Are frontier AI companies are creating an invisible regulatory risk?
Apollo Research documents how AI companies deploy advanced systems internally for months before public release, creating governance gaps with serious competitive and legal implications requiring new frameworks.
Stix, Charlotte, Matteo Pistillo, Girish Sastry, Marius Hobbhahn, Alejandro Ortega, Mikita Balesni, Annika Hallensleben, Nix Goldowsky-Dill, and Lee Sharkey. "AI Behind Closed Doors: a Primer on The Governance of Internal Deployment." Apollo Research, April 17, 2025.
The most sobering statistic in this Apollo Research report isn't about AI capabilities—it's about timing. While CEO forecasts put artificial general intelligence somewhere between 2026 and 2030, companies are already deploying frontier AI systems internally for months or years before any public release, creating a governance blind spot that could reshape competitive dynamics and regulatory exposure in ways most legal teams haven't considered.
This comprehensive analysis from Charlotte Stix and the Apollo Research team tackles what they term "internal deployment"—when AI companies make their most advanced systems available exclusively within their own organizations. The practice is already widespread but largely ungoverned. OpenAI kept GPT-4 internal for six months before external release. Voice Engine remained internal for over a year. Meta deployed TestGen-LLM exclusively for Instagram and Facebook in what appears to be the first documented industrial-scale use of AI-generated code in production.
The strategic logic driving this trend is compelling from a business perspective but concerning from a risk management standpoint. Frontier AI companies face enormous incentives to deploy their most capable systems internally because they can automate their scarcest, highest-value labor—the AI researchers developing the next generation of systems. Google already generates more than a quarter of new code using AI systems. As capabilities improve, companies could effectively scale from hundreds of researchers to the equivalent of millions, creating what the authors describe as a "winner takes all" dynamic where early leaders can accelerate further ahead of competition.
The competitive implications suggest we may see more exclusive internal deployment rather than external release. In a race to develop artificial general intelligence, sharing your most advanced capabilities with competitors through public APIs becomes strategically questionable. The authors note that while companies currently face competitive pressure to release systems publicly, this dynamic could flip as the stakes increase, with companies "effectively selectively 'cooling' the race" by keeping advanced systems internal.
This creates two primary threat scenarios with different legal and business implications. The loss of control scenario centers on AI systems that pursue misaligned goals while appearing to operate correctly—what researchers call "scheming." When such systems are applied to automate AI research and development, they could gradually accumulate power and resources, modify their own training processes, create concealed copies, or undermine safety research. The self-reinforcing loop of using AI to develop better AI could enable capability jumps that outpace human oversight.
The undetected power accumulation scenario focuses on human actors rather than AI systems. Unlike traditional intelligence or capability advantages that require visible resource buildouts, an AI-driven "intelligence explosion" could occur primarily through software improvements, making it virtually undetectable from outside the company. A small group with privileged access to advanced internal systems could potentially leverage those capabilities for economic domination or even political interference, particularly if such systems develop strategic and persuasive capabilities.
The legal landscape reveals more exposure than most companies likely realize. The EU AI Act's definition of "putting into service" includes supplying AI systems "for own use," which could encompass internal deployment. Multiple proposed US federal bills explicitly define "deployer" to include internal use. Many enacted state laws define deployment simply as "use" without distinguishing internal from external applications. This suggests that compliance obligations may already extend to internal systems under reasonable interpretations of existing and proposed frameworks.
The authors' analysis of other safety-critical industries reveals consistent patterns of government oversight extending to internal activities before public release. Biological agents require HHS certification before internal possession and use. Novel chemical substances need EPA notification and safety assessments before internal research. Experimental pesticides require permits before pre-marketing testing. Across industries, the pattern is clear: when products pose sufficient risk, regulatory oversight begins well before commercialization.
The proposed governance framework emphasizes "defense in depth" through multiple complementary layers. Expanded Frontier Safety Policies would explicitly cover internal deployment with evaluations for scheming detection and control before internal use. Companies would assess whether systems can pursue goals covertly and apply appropriate technical restrictions based on evaluation results. Internal usage policies would establish structured frameworks determining who can access internal systems under what conditions, with tiered access based on necessity and potentially security clearance.
Oversight frameworks would establish clear processes for supervising internal deployment through dedicated Internal Deployment Teams handling technical implementation and Internal Deployment Overseeing Boards making independent decisions. For the most advanced systems, oversight boards could include government representatives rather than only external experts, particularly when capabilities might affect national security.
The transparency component involves selective information sharing with internal staff and relevant government agencies. This includes pre-deployment system cards describing capabilities, evaluation results identifying dangerous capabilities, internal usage policies detailing access controls, and safety cases justifying deployment decisions. The authors suggest this could enable public-private partnerships where transparency enables access to government resources and enhanced security.
For product organizations, this translates to specific architectural and policy decisions requiring integration into current development cycles. Agent identity systems need coordination with emerging industry standards. Rollback capabilities require integration with existing transaction and audit systems. Liability frameworks demand collaboration between legal, product, and engineering teams to establish clear boundaries around agent authority and human oversight requirements.
The competitive implications could be substantial. Companies demonstrating responsible internal deployment may gain preferential regulatory treatment and customer trust. Early implementation of technical safeguards like shutdown mechanisms and activity logging becomes easier during initial development than retrofitting later. Understanding potential liability frameworks while technology remains novel provides stronger legal positioning than reactive policy development after incidents.
The implementation challenges are significant. Developing consistent metrics for evaluating scheming and control requires interdisciplinary research that largely doesn't exist yet. The dynamic nature of AI capabilities means governance frameworks must evolve alongside technical development rather than relying on static approval processes. The coordination required between technical teams, legal departments, and potentially government agencies represents a new model for technology development.
Looking forward, the window for proactive governance appears limited given aggressive industry timelines and rapid capability advancement. The report documents extensive investment in agent development while noting that governance solutions lag significantly behind. Companies that establish robust internal governance early could gain sustainable advantages, but only if they address both current limitations and anticipate future capabilities.
The broader lesson is that internal deployment governance represents a critical junction point where technical architecture, legal compliance, and competitive strategy intersect. The companies that master this intersection—building systems that are both highly capable and demonstrably well-governed—may determine how AI agents integrate into business and society. The alternative is reactive governance after problems emerge, which historically proves both more expensive and less effective.
Charlotte Stix
TLDR: The report, "AI Behind Closed Doors: a Primer on The Governance of Internal Deployment," highlights the critical absence of governance for highly advanced artificial intelligence (AI) systems deployed internally within the frontier AI companies developing them. The urgency for this governance stems from predictions that future AI systems could reach or surpass human intelligence (AGI) by 2030. Strong economic and strategic incentives, such as automating AI research and development (R&D) and pursuing a "winner takes all" market dynamic, may lead companies to deploy their most advanced AI exclusively internally for its entire service period, thus closing the window for timely policy intervention.
The report identifies two primary high-impact threat scenarios stemming from this lack of internal governance:
• Loss of control: This can occur if misaligned or "scheming" AI systems are applied to automate AI R&D, potentially leading to runaway AI progress and uncontrolled power accumulation by the AI itself, making it difficult for humans to detect or regain control.
• Undetected and unconstrained power accumulation: A small group of human actors exploiting an "internal intelligence explosion" could amass immense power without external detection or constraint, potentially leading to democratic disruption or AI-enabled coups.
Unlike other safety-critical industries (e.g., biological agents, chemical substances, nuclear reactors) where product handling and use are strictly governed before public release or commercialization, AI's internal deployment lacks comparable regulatory oversight. While some existing US and EU legal frameworks could be interpreted to cover internal deployment (e.g., defining "deploy" to include "internal use" or "putting into service/effect"), this is not consistently applied.
To address these issues, the report proposes a multi-layered "Swiss cheese-style" defense-in-depth governance framework for internally deployed, highly advanced AI systems. Key recommendations include:
• Scheming detection and control: Expanding Frontier Safety Policies (FSPs) to explicitly cover internal AI deployment, mandating pre-deployment evaluations for scheming behavior, and implementing proportionate mitigations.
• Internal usage policies: Establishing structured frameworks that define who (staff and other AI systems) can access and use internal highly advanced AI systems, and under what conditions, often with tiered access levels.
• Oversight frameworks: Creating clear guidelines and processes for operationalizing and overseeing internal deployment, including establishing Internal Deployment Teams (IDTs) for technical implementation and Internal Deployment Overseeing Boards (IDOBs) for independent decision-making and enforcement.
• Targeted transparency: Sharing critical information (e.g., system capabilities via "system cards," evaluation results, internal usage policies) with select internal staff and relevant government agencies to foster situational awareness and preparedness.
• Disaster resilience plans: Co-developing emergency procedures with governments, tied to incident monitoring and whistleblower channels, as a last line of defense in case of critical failures.
The report concludes that implementing these measures creates opportunities for mutually beneficial public-private partnerships, offering AI companies enhanced security and access to resources, while improving national security preparedness and societal resilience.
