Your AI Is "In Production." That Doesn’t Mean It’s Production-Ready.

The rapid deployment of AI-powered features into production environments often mirrors the simplicity of launching a static landing page: a code merge, a celebratory demo, and then, frequently, an abrupt encounter with reality. This article delves into the critical gap between AI feature deployment and true production readiness, introducing the AI Production Readiness Framework (APRF) as a potential solution for engineers grappling with the complexities of operationalizing AI. APRF, a vendor-neutral framework, emphasizes a "gated" methodology, ensuring that AI systems undergo rigorous, machine-readable checks before and during their operational lifecycle.

The current approach to deploying AI, particularly Large Language Model (LLM) features, is frequently characterized by a lack of robust production controls. This often leads to failures that are not attributed to the inherent intelligence of the model itself, but rather to the absence of production-grade safeguards. These failures can manifest in various forms, including unexpected system behavior, security vulnerabilities, and performance degradation. The core issue, as highlighted by the APRF initiative, is the deployment of "production systems without production gates."

While existing frameworks like the NIST AI Risk Management Framework (RMF), ISO/IEC 42001, and SOC 2 provide valuable guidance on thinking about risk, managing AI systems, and building organizational trust, they often fall short for the engineer directly responsible for on-call duties. The fundamental question that remains largely unanswered by these frameworks for the frontline developer is starkly pragmatic: "Can this AI application safely operate in production?" This is the central tenet driving the development and proposal of the AI Production Readiness Framework (APRF).

APRF is not positioned as a certification, a partner network, or a simple numerical score for board presentations. Instead, it functions as a strictly "gated" methodology. This means that mandatory checks must be passed; failure to do so acts as an immediate blocker to deployment or continued operation. Recommended controls, while important, do not contribute to an overall score and are not averaged into the critical gate-passing criteria. This approach is designed to prevent the dilution of critical safety and security measures, a common pitfall when systems rely on averaged scores that can mask significant underlying risks.

The distinction between a "vanity score" and a true "gate result" is crucial. A vanity score might average all controls, potentially masking the absence of a critical kill switch. In contrast, APRF’s gate result is determined by the logical AND of all mandatory checks; a single failure results in a blocked release. Similarly, capability attainment is assessed as the minimum across all relevant pillars, not an average, ensuring that the weakest link dictates the overall readiness level. This strictness, the framework argues, is a necessary reflection of the unforgiving nature of production environments.

The Mental Model: Demo Path vs. Gated Path

To understand the shift APRF advocates, it’s useful to contrast the typical "demo path" with the proposed "APRF path." The demo path often involves a straightforward progression: a prompt works, the code is merged, and the feature is shipped. The APRF path, however, introduces a crucial intermediate step: after the prompt works and the PR is merged, mandatory checks must be executed. Only if all these gates pass is the system attested and shipped. Post-deployment, continuous observation and drilling are essential components of this lifecycle.

This structured approach aims to prevent the common scenario where a feature is deemed "ready" based on limited testing or a successful initial demonstration, only to encounter unforeseen issues in the complexities of real-world usage. The APRF emphasizes that production readiness is not a one-time achievement but an ongoing process validated by continuous adherence to predefined gates.

The APRF Catalog: Domains, Pillars, and Profiles

As of its v0.10 release, the APRF catalog is structured around eight core domains: Security, Safety, Data, Model Lifecycle, Agents, Reliability, Cost, and Governance. Within these domains, 27 distinct pillars focus on specific control areas. The framework offers different profiles tailored to the criticality and regulatory environment of the AI system. The "Core Profile" defines approximately 40 gates for Tier-2 customer-facing AI applications, while the "Regulated Profile" expands this to 61 gates for Tier-3 or regulated systems.

Additionally, "Lenses" provide extra mandatory checks for specific AI capabilities, such as Retrieval Augmented Generation (RAG), Agents, Voice, and Coding agents. The framework also includes machine-readable specifications in JSON format and downloadable self-attestation documents. Crucially, APRF provides crosswalks to established standards like NIST AI RMF, ISO 42001, OWASP LLM Top 10, SOC 2, AWS Well-Architected Framework, and SLSA. These crosswalks are intended for informative alignment only, underscoring that APRF is not a replacement for but a complementary framework focused on operational readiness.

A Concrete Example: Agents with Tools

To illustrate APRF’s practical application, consider an AI agent equipped with tools. Within the scope of the Core profile and the Agents lens, several mandatory gates (gates denoted with "M") apply. For instance, TOL-M1 requires that tool calls are authorized server-side, not solely based on model output. This prevents the model from directly executing arbitrary commands. TOL-M2 mandates a per-agent tool allowlist, ensuring that only explicitly permitted tools can be invoked. TOL-M3 elevates the importance of high-impact tools, requiring them to be protected by approval processes, dual control mechanisms, or explicit policies. Similarly, HUM-M1 ensures that high-impact actions performed by the system are inventoried and gated. Cost-related gates, such as step budgets, kill switches, and spend ceilings, are also critical for managing the operational expenses of AI systems.

The consequence of failing to demonstrate compliance with these gates is direct: a "gate fail," not a compromised readiness score. This is reinforced by the "model proposes, platform disposes" diagram, which visually represents the flow where the agent runtime proposes tool calls, but the tool gateway rigorously validates these proposals against allowlists, schema requirements, and, for high-impact tools, explicit approval workflows. Any deviation from these checks results in denial or further gating.

Practical Implementation: Integrating APRF Principles This Week

The adoption of APRF principles can begin immediately without a full organizational overhaul. Developers can integrate these concepts into their existing workflows.

1. Tool Allowlist and Schema Validation: Implementing a system where tools are explicitly allowlisted and their arguments are validated against predefined schemas is a fundamental step. This can be achieved using libraries like Zod in TypeScript, as demonstrated in the provided code snippet. The invokeTool function encapsulates this logic, first checking the allowlist, then validating the arguments against the tool’s schema, and finally enforcing approval requirements for high-impact tools. This ensures that the agent can only interact with authorized tools and that those interactions conform to expected parameters.

   

2. Non-Bypassable Approval: A critical aspect of production readiness is ensuring that approval mechanisms cannot be circumvented. The Python sketch for execute_tool illustrates how to enforce this. If a tool is categorized as "HIGH_IMPACT," its execution is contingent on a valid approval_id that corresponds to an "approved" decision for that specific tool and arguments. A test case using curl is provided to verify that attempts to bypass this approval mechanism by directly calling a high-impact tool endpoint result in errors like TOOL_DENIED or 403 Forbidden, rather than silently executing the action.

3. YAML Policy Pack: Versioning a policy pack in YAML alongside the application code provides a clear declaration of adherence to APRF. This policy.yaml file specifies the APRF version, the chosen profile, criticality level, and any applicable lenses. It then maps each mandatory gate (check ID) to the specific evidence that proves its fulfillment, such as test files, configuration files, or documentation. This creates a traceable link between the requirements of the framework and the implementation details of the system.

4. CI/CD Integration: Automating the verification of these policies within CI/CD pipelines is essential. A GitHub Actions workflow (aprf-gates.yml) can be configured to fetch the APRF specification, run unit and contract tests for the tool gateway, and execute a script that checks for the existence of evidence files declared in the policy.yaml. This ensures that every pull request or push to the main branch is validated against the defined gates, preventing unverified code from reaching production.

5. Attestation JSON: The output of a successful APRF assessment is a machine-readable attestation JSON. This document serves as a reproducible artifact for auditing, change management, and internal reporting. It details the assessed system, the APRF version, the specific gates that were evaluated, whether they passed or failed, and references to the evidence. A failed mandatory gate in the attestation directly translates to a "gate fail" for the system, with explicit identification of the blockers. This is a departure from systems that might report a partial "readiness" score.

Reference Assessment and Target Audience

StackRail has made a reference assessment available, allowing teams to evaluate their AI systems against the Core and Regulated profiles, with optional lenses. Completing this assessment generates an attestation JSON, providing a tangible output of the evaluation process.

APRF is primarily intended for engineering teams responsible for deploying and operating AI systems in production. This includes ML engineers, platform engineers, SREs, and security engineers who need to ensure the reliability, safety, and security of AI applications. It is particularly relevant for organizations handling sensitive data, operating in regulated industries, or deploying AI systems that have significant downstream impacts on users or business operations.

Conversely, APRF is not designed for entities seeking a broad, easily obtainable compliance badge that broadly declares "we’re compliant with everything." The framework’s strength lies in its specificity and its refusal to obscure critical failures with aggregate scores. APRF aims to be an honest assessment of an AI system’s readiness for production, acknowledging that true readiness requires rigorous, verifiable adherence to predefined safety and operational standards.

The APRF is currently a working draft, with StackRail as the initial publisher. The long-term intention is for the framework to be stewarded by a neutral working group through public RFC processes, fostering broader community contribution and evolution. Interested parties are encouraged to contribute to the development at stackrail.io/aprf/rfc.