Cost-Aware Prompt Engineering: A Governance Architecture for Optimizing Enterprise AI Agents

Abstract

As enterprises deploy large language model (LLM)-based and agentic AI systems across critical business functions, prompt engineering has emerged as a significant operational bottleneck. Prompts that are poorly structured, excessively verbose, or manually tuned without systematic comparison introduce hidden costs: elevated token consumption, increased inference latency, inconsistent output quality, and deployment risk. Despite its importance, prompt engineering in most enterprise environments remains an informal, trial-and-error discipline without governance, reproducibility, or measurable quality criteria. This paper proposes a Cost-Aware Prompt Engineering Governance Architecture (CAPEGA) designed for enterprise production environments. The architecture integrates four core components: iterative prompt optimization using SELF-REFINE techniques, structured prompt compression with instruction-preservation controls, metric-driven prompt comparison with cost and latency artifact generation, and CI/CD-ready governance workflows for prompt promotion and rollback. CAPEGA's contribution is architectural integration rather than novel algorithmic techniques: it combines established optimization methods into a unified, production-ready governance pipeline. Together, these components transform prompt engineering from an ad hoc craft into a reproducible, auditable engineering discipline. The framework is model-agnostic in design, model-specific in execution, and intended for integration into existing enterprise AI release pipelines. By establishing measurable quality gates for prompts, CAPEGA enables organizations to reduce token overhead, improve output consistency, control inference cost, and generate audit-ready evidence for production deployment decisions. An implementation architecture with component-level design is presented alongside the conceptual framework.