Unlock AI Potential with Cloudflare AI Gateway

The rapid proliferation of Artificial Intelligence, particularly in the realm of Large Language Models (LLMs) and generative AI, has ushered in an era of unprecedented innovation. From automating customer service interactions to generating creative content and sophisticated code, AI's transformative power is undeniable. However, integrating these advanced AI capabilities into existing applications and enterprise workflows is far from straightforward. Developers and organizations frequently encounter a myriad of challenges, including managing diverse API specifications, ensuring data security, optimizing performance, controlling costs, and maintaining observability across a fragmented AI landscape. This is where an intelligent AI Gateway becomes not just beneficial, but absolutely essential. It acts as the critical intermediary, simplifying the complex dance between your applications and the myriad of AI models available today.

Cloudflare, renowned for its global network and robust internet infrastructure services, has stepped into this breach with its innovative Cloudflare AI Gateway. This powerful solution is designed to abstract away the complexities of AI integration, offering a unified, secure, and performant layer for interacting with various AI models. By leveraging Cloudflare's extensive edge network, the AI Gateway promises to unlock the full potential of AI for businesses of all sizes, making AI adoption more accessible, cost-effective, and resilient. This article will delve deep into the mechanics, benefits, and transformative capabilities of the Cloudflare AI Gateway, exploring how it streamlines the deployment of AI, enhances security postures, and provides the crucial insights needed to effectively manage AI resources, ultimately paving the way for a more integrated and efficient AI-powered future.

The AI Revolution and Its Intricate Challenges

The current wave of AI, spearheaded by advancements in machine learning and particularly Large Language Models, is more than just a technological evolution; it's a paradigm shift. We've moved beyond theoretical discussions into practical applications that are reshaping industries from healthcare to finance, entertainment to manufacturing. Generative AI models, capable of producing human-like text, images, audio, and even video, have captured the public imagination and demonstrated a staggering array of potential use cases. Businesses are now keenly aware that integrating AI is no longer an option but a strategic imperative to remain competitive and foster innovation. However, the path to AI adoption is fraught with significant hurdles, often more complex than initially perceived.

One of the foremost challenges lies in the sheer diversity and rapid evolution of AI models. The landscape is a constantly shifting mosaic of providers—OpenAI, Google, Anthropic, Hugging Face, and countless others—each offering unique models with distinct APIs, authentication mechanisms, rate limits, and data formats. Integrating even a handful of these models directly into an application can lead to an unwieldy tangle of custom code, each piece brittle and susceptible to breakage with every model update. This direct integration approach not only inflates development time and cost but also creates substantial technical debt, making it difficult to switch models or even update existing ones without extensive refactoring. The dream of leveraging the "best" model for a specific task quickly devolves into a nightmare of integration complexity.

Beyond the technical fragmentation, security concerns loom large. When applications interact with external AI models, sensitive data often traverses the internet. This raises critical questions about data privacy, compliance with regulations like GDPR and CCPA, and the potential for data leakage. Furthermore, the burgeoning field of prompt engineering has highlighted new attack vectors, such as "prompt injection," where malicious inputs can trick AI models into divulging confidential information, generating harmful content, or performing unintended actions. Traditional security measures, designed for REST APIs, often fall short when confronted with the nuanced vulnerabilities inherent in AI model interactions. Without a robust security layer, organizations risk not only reputational damage but also severe financial penalties and legal repercussions.

Performance and reliability are equally pressing concerns. AI models, especially LLMs, can be resource-intensive, leading to variable response times and potential bottlenecks. Applications relying on AI must ensure low latency for a smooth user experience, particularly in real-time interactions like chatbots or voice assistants. Managing consistent performance across various AI providers, some of which may experience outages or degradation, requires sophisticated traffic management and fallback strategies. Moreover, the ability to scale these AI interactions to accommodate fluctuating demand—from a handful of requests per second to thousands—without significant infrastructure overhaul is a non-trivial engineering feat.

Cost management presents another significant obstacle. AI model usage is typically billed based on tokens, computational time, or API calls, and these costs can quickly spiral out of control if not carefully monitored and optimized. Without a centralized mechanism to track usage, set budgets, and implement cost-saving measures like caching, organizations can find themselves blindsided by unexpectedly high bills. The lack of transparency into AI consumption across different teams or applications makes it nearly impossible to allocate costs accurately or identify areas for optimization.

Finally, the specter of vendor lock-in looms large. Committing heavily to a single AI provider's ecosystem, from their specific API formats to their proprietary model weights, can severely limit an organization's flexibility. If a better, more cost-effective, or more specialized model emerges from a different provider, or if the current provider raises prices or changes terms, migrating away can be a daunting and expensive endeavor. This lack of interoperability stifles innovation and forces businesses into uncomfortable strategic compromises. These multifaceted challenges underscore the urgent need for a sophisticated, unified solution—a dedicated AI Gateway—that can mediate, secure, and optimize all interactions with AI models, allowing developers to focus on building innovative applications rather than wrestling with integration complexities.

Understanding the Concept of an AI Gateway

At its core, an AI Gateway represents a specialized evolution of the traditional API Gateway, meticulously engineered to address the unique demands and complexities of interacting with Artificial Intelligence models. While a conventional API Gateway acts as a single entry point for managing and routing requests to various backend services, an AI Gateway extends this functionality by providing AI-specific features that are crucial for efficient, secure, and scalable AI integration. It sits strategically between your client applications and the diverse landscape of AI models, serving as an intelligent proxy that simplifies, enhances, and secures every interaction.

The fundamental purpose of an AI Gateway is to abstract away the underlying heterogeneity of AI services. Imagine a scenario where your application needs to use an LLM for text generation, a vision model for image analysis, and a speech-to-text model for audio processing. Each of these models, likely from different providers, will have its own API endpoint, authentication scheme, data format, and pricing structure. Without an AI Gateway, your application would need to implement custom logic for each, leading to bloated code, increased maintenance overhead, and a steep learning curve for developers. The AI Gateway solves this by presenting a unified, standardized interface to your applications, regardless of the backend AI model being invoked. This means developers interact with a single, consistent API, and the gateway handles the translation, routing, and management of requests to the appropriate AI service.

One of the most critical aspects of an AI Gateway is its role as an LLM Gateway. Large Language Models (LLMs) like GPT-4, Llama, and Claude are incredibly powerful but come with specific operational challenges. They are often resource-intensive, leading to higher latency and costs, and their output can be sensitive to subtle changes in prompts. An LLM Gateway specifically optimizes interactions with these models by offering features such as:

• Prompt Management and Versioning: Allowing developers to store, version, and A/B test different prompts, ensuring consistency and enabling quick iteration without modifying application code.
• Response Caching: Significantly reducing latency and cost for repeated queries by storing and serving previously generated responses. This is especially vital for LLMs where token usage directly translates to cost.
• Model Routing and Fallbacks: Dynamically routing requests to the best-performing or most cost-effective LLM based on criteria like availability, performance, or specific prompt requirements. It also provides automatic fallbacks to alternative models if a primary one becomes unavailable, ensuring service continuity.
• Input/Output Transformation: Standardizing the request and response formats across various LLMs, allowing applications to seamlessly switch between models without breaking their own integration logic. For instance, if OpenAI changes its API, the LLM Gateway can adapt its internal logic, leaving your application untouched.

Beyond LLM-specific features, a comprehensive AI Gateway typically incorporates a suite of functionalities borrowed and enhanced from traditional API Gateway design, tailored for AI workloads:

• Routing and Load Balancing: Directing incoming requests to the most appropriate AI model or instance, distributing traffic efficiently to prevent overload and ensure optimal performance.
• Authentication and Authorization: Securing access to AI models by enforcing robust authentication mechanisms (e.g., API keys, OAuth, JWT) and fine-grained authorization policies, ensuring only authorized applications and users can invoke specific AI services.
• Rate Limiting: Protecting AI models from abuse and ensuring fair usage by restricting the number of requests an application or user can make within a given time frame. This prevents runaway costs and service degradation.
• Observability and Analytics: Providing a centralized hub for monitoring all AI interactions. This includes detailed logging of requests and responses, performance metrics (latency, error rates), and usage analytics, offering invaluable insights into AI consumption, model performance, and potential issues. This data is critical for cost management, performance tuning, and compliance.
• Security: Implementing AI-specific security measures, such as input sanitization to mitigate prompt injection attacks, sensitive data redaction before sending to models, and integration with broader security services like Web Application Firewalls (WAFs) and DDoS protection.
• Caching: Beyond LLM-specific response caching, caching can also apply to embeddings or other computationally expensive AI outputs, reducing redundant computations and improving overall efficiency.
• Cost Management: By centralizing usage tracking and applying caching and intelligent routing strategies, an AI Gateway provides a clear picture of AI expenditure and enables proactive cost optimization.

In essence, an AI Gateway transforms the chaotic and disparate world of AI models into a manageable, secure, and performant ecosystem. It empowers developers to rapidly build and deploy AI-powered applications without getting bogged down in low-level integration details, while simultaneously providing enterprises with the control, visibility, and security necessary to confidently scale their AI initiatives. It's the silent orchestrator that makes the promise of integrated AI a tangible reality, allowing businesses to truly focus on innovation rather than infrastructure.

Deep Dive into Cloudflare AI Gateway

Cloudflare AI Gateway emerges as a sophisticated solution meticulously crafted to tackle the intricate challenges of AI integration, security, and performance at scale. It leverages Cloudflare's sprawling global network, a distinctive advantage that positions it as a highly effective AI Gateway and LLM Gateway. By acting as a intelligent intermediary at the edge, Cloudflare AI Gateway streamlines the interaction between your applications and a diverse array of AI models, including popular LLMs, vision models, and other specialized AI services. This architectural approach not only simplifies the developer experience but also significantly enhances the operational aspects of AI deployment.

Architecture and Integration

The core strength of Cloudflare AI Gateway lies in its distributed architecture, which is deeply interwoven with Cloudflare's global network spanning hundreds of cities worldwide. When a client application makes a request to an AI model through the Cloudflare AI Gateway, that request first hits the nearest Cloudflare edge location. This geographical proximity to users dramatically reduces latency, providing a snappier and more responsive user experience for AI-powered applications.

From the edge, the AI Gateway intelligently routes the request to the appropriate backend AI service. This routing is not just a simple proxy; it can involve complex logic based on load, model availability, cost considerations, or even specific user groups. The gateway is designed to be highly flexible, integrating seamlessly with a wide range of AI providers, including OpenAI, Google Gemini, Hugging Face models, and many others. This allows organizations to maintain a vendor-agnostic approach, avoiding lock-in and enabling them to select the best-of-breed models for their specific needs without rewriting application code.

Integration with existing infrastructure is engineered for simplicity. Developers typically point their AI requests to a Cloudflare-managed endpoint, and the AI Gateway handles the rest. This minimal configuration effort means that even applications currently making direct API calls to AI providers can be transitioned to use the gateway with relatively minor code changes. Furthermore, the AI Gateway can be integrated with other Cloudflare services, such as Cloudflare Workers for custom logic processing at the edge, Cloudflare WAF for enhanced security, and Cloudflare R2 for storing and managing AI-related data, creating a holistic and powerful AI operational environment.

Core Features and Benefits

The Cloudflare AI Gateway is packed with features designed to optimize every facet of AI interaction:

1. Unified API Access

One of the most immediate benefits of the Cloudflare AI Gateway is its ability to provide unified API access to a multitude of AI models. Instead of managing separate API keys, endpoints, and data formats for OpenAI, Hugging Face, or custom-trained models, your applications interact with a single, consistent interface provided by the AI Gateway. * Simplified Development: Developers write code once against the gateway's standardized API, significantly reducing development time and complexity. They don't need to learn the nuances of each AI provider's SDK or API specification. * Interoperability: Easily switch between different AI models or providers without requiring application-level code changes. This flexibility is crucial for A/B testing models, optimizing for cost or performance, or migrating away from a provider if terms change. * Centralized Control: All AI interactions flow through a single point, enabling centralized management of authentication, permissions, and routing rules.

2. Caching for Performance and Cost Savings

Caching is a cornerstone feature of the Cloudflare AI Gateway, offering substantial benefits in terms of both performance and cost reduction, especially pertinent for LLM Gateway operations. * Reduced Latency: For identical or highly similar requests, the AI Gateway can serve responses directly from its cache at the edge. This bypasses the need to send requests to the origin AI model, drastically cutting down response times, particularly for users located far from the AI provider's data centers. This is invaluable for real-time applications where every millisecond counts. * Cost Optimization: Many AI models, particularly LLMs, are billed per token or per API call. By serving cached responses, the gateway eliminates redundant calls to the backend AI service, directly translating into significant cost savings. Organizations can configure caching policies based on specific needs, such as caching popular queries for a certain duration. * Reduced Load on Origin AI: Caching offloads a substantial portion of traffic from the backend AI models, ensuring they are not unnecessarily burdened by repetitive requests. This can improve the reliability and response times of the origin AI service for unique, uncached requests.

3. Rate Limiting and Load Balancing

These features are essential for maintaining the stability, fairness, and cost-effectiveness of AI model usage. * Preventing Abuse and Overload: Rate limiting allows organizations to define strict policies on the number of requests an individual user, application, or API key can make within a specified timeframe. This prevents malicious actors from overwhelming AI models (DDoS-like attacks) and safeguards against accidental runaway usage that could incur massive costs. * Ensuring Fair Access: By enforcing rate limits, the gateway ensures that all users or applications have equitable access to AI resources, preventing any single entity from monopolizing capacity. * Optimized Resource Utilization: Load balancing intelligently distributes incoming AI requests across multiple instances of an AI model or even across different AI providers. This optimizes resource utilization, minimizes latency by directing traffic to the least busy or geographically closest instance, and enhances resilience by ensuring that if one AI endpoint fails, traffic is seamlessly rerouted to healthy alternatives.

4. Observability and Analytics

Visibility into AI usage is paramount for effective management, cost control, and performance optimization. The Cloudflare AI Gateway provides comprehensive observability features. * Detailed Logging: Every request and response passing through the gateway is logged, providing granular data on who made the request, which AI model was invoked, the prompt used (with privacy controls), the response received, latency, and any errors. This detailed logging is invaluable for debugging, auditing, and compliance. * Real-time Metrics and Dashboards: Organizations gain access to real-time metrics on AI usage, including total requests, cached requests, error rates, average latency, and token consumption. These metrics are presented in intuitive dashboards, allowing teams to quickly assess the health and performance of their AI integrations. * Cost Tracking and Allocation: With detailed usage data, businesses can precisely track costs associated with different AI models, applications, or even individual users. This enables accurate cost allocation to different departments or projects and helps identify opportunities for cost reduction. * Performance Monitoring: Teams can monitor the performance of various AI models over time, identifying bottlenecks, performance degradations, or inefficiencies that might warrant switching to a different model or optimizing prompts.

5. Security Features

Security is a foundational pillar of the Cloudflare AI Gateway, designed to protect sensitive data and prevent AI-specific vulnerabilities. * Prompt Logging Control: A critical feature for data privacy, the AI Gateway allows organizations to control whether prompts and responses are logged, and for how long. For sensitive applications, prompts can be redacted or not logged at all, ensuring that confidential information doesn't persist unnecessarily in logs. * Data Masking and Redaction: Implement policies to automatically identify and mask or redact sensitive information (e.g., PII, financial data) from prompts before they are sent to AI models, and from responses before they are returned to the application. * WAF Integration: As part of the broader Cloudflare ecosystem, the AI Gateway benefits from Cloudflare's industry-leading Web Application Firewall (WAF), which can protect against common web vulnerabilities and AI-specific attacks like prompt injection (by identifying and blocking suspicious input patterns). * DDoS Protection: Leveraging Cloudflare's massive network, the AI Gateway is inherently protected against volumetric DDoS attacks, ensuring the availability of AI services even under hostile conditions. * Authentication and Authorization: Enforce strong authentication mechanisms (e.g., API keys, mTLS, Cloudflare Access) to ensure only authorized entities can access AI models. Fine-grained authorization controls allow for precise management of which applications or users can invoke specific models or perform certain actions.

6. Cost Management

Beyond caching, the AI Gateway offers advanced capabilities to manage and optimize AI expenditure. * Budget Alerts: Set up alerts to notify administrators when AI usage approaches predefined budget thresholds, preventing unexpected cost overruns. * Tiered Access: Implement tiered access to different AI models based on cost or performance. For example, less critical applications might be routed to a cheaper, lower-performance model, while premium applications use a more expensive, higher-performance alternative. * Spend Optimization Analytics: Leverage detailed analytics to identify areas where AI spending can be optimized, perhaps by refining prompts, adjusting caching strategies, or switching to more cost-effective models for specific workloads.

7. Flexibility and Vendor Agnosticism

The AI Gateway's design prioritizes flexibility, enabling businesses to adapt to the rapidly changing AI landscape. * Model Agnostic: Support for a wide array of AI models from different providers means organizations are not locked into a single vendor. This fosters innovation and allows for strategic choices based on model capabilities, pricing, and compliance requirements. * Dynamic Model Switching: Easily switch between models (e.g., from GPT-3.5 to GPT-4, or even to an open-source alternative) through configuration changes in the gateway, without requiring application code modifications. This is invaluable for testing new models or responding to market changes.

8. Edge Computing Benefits

Cloudflare's unique position at the network edge offers distinct advantages for AI workloads. * Reduced Latency: By placing the AI Gateway physically closer to end-users, the round-trip time for requests and responses is minimized, leading to faster application performance. This is particularly impactful for interactive AI experiences. * Enhanced Reliability: Distributing the gateway across a global network means there's no single point of failure. If one edge location experiences issues, traffic can be seamlessly rerouted to another, ensuring high availability of AI services. * Localized Processing: In the future, as edge computing capabilities expand, certain AI inference tasks might even be performed directly at the edge, further reducing latency and potentially enabling new classes of AI applications.

The Cloudflare AI Gateway is more than just a proxy; it's a comprehensive platform that addresses the core operational challenges of AI adoption. By centralizing management, bolstering security, optimizing performance and cost, and providing deep observability, it empowers organizations to confidently integrate and scale AI into their products and services, truly unlocking the potential of this transformative technology.

Practical Use Cases and Scenarios

The versatility of the Cloudflare AI Gateway makes it an indispensable tool across a broad spectrum of practical use cases, simplifying complex AI integrations and enhancing existing workflows. Its capabilities extend far beyond mere proxying, offering strategic advantages for businesses striving to harness the full power of artificial intelligence.

Building AI-Powered Applications

For developers creating new AI-driven applications, the Cloudflare AI Gateway acts as a foundational layer that accelerates development and ensures robustness. Consider a startup building an advanced chatbot for customer support, a content platform generating articles using LLMs, or a creative agency designing tools for AI-driven image manipulation. * Chatbots and Virtual Assistants: A conversational AI application often needs to integrate with multiple LLMs for different purposes – one for general knowledge, another for specific domain expertise, and perhaps a third for sentiment analysis. The AI Gateway allows the chatbot to interact with a single endpoint, and the gateway intelligently routes the prompt to the most suitable backend model. Caching ensures that repetitive "hello" or "thank you" responses don't incur repeated LLM costs, and rate limiting prevents a single user from overwhelming the system or driving up costs. Observability provides insights into which prompts are most common, which models are performing best, and where users might be getting stuck. * Content Generation Platforms: A platform that generates marketing copy, blog posts, or code snippets might leverage different LLMs based on the desired tone, length, or subject matter. The AI Gateway provides a unified interface, allowing the application to simply request "generate a blog post about X" and the gateway dynamically routes to the appropriate LLM. Prompt versioning through the gateway enables A/B testing of different prompts to determine which generates the highest quality content without modifying the core application logic. * Data Analysis and Summarization Tools: Applications that process large volumes of text data for summarization, entity extraction, or trend analysis can benefit from the gateway's ability to abstract model complexities. Developers can easily swap out underlying summarization models or integrate new ones as they emerge, always presenting a consistent API to their users. Security features like data masking ensure sensitive information isn't exposed to third-party AI models unnecessarily.

Enterprise AI Adoption

Large enterprises integrating AI into existing, complex workflows face challenges of scale, security, and compliance. The Cloudflare AI Gateway is perfectly suited to meet these rigorous demands. * Integrating AI into CRM/ERP Systems: Enterprises can enhance their customer relationship management (CRM) systems with AI capabilities like intelligent lead qualification, automated email responses, or personalized recommendations. The AI Gateway acts as a secure and controlled bridge between the CRM and external AI models. For example, customer queries coming into the CRM can be routed through the gateway to an LLM for intent classification, and the gateway's prompt logging controls ensure that customer data handling complies with internal policies and external regulations. * Automating Business Processes: AI can automate tasks like invoice processing (using vision AI for OCR and LLMs for data extraction), legal document review, or HR query resolution. By channeling these AI interactions through the gateway, enterprises gain centralized control over access, ensure data privacy with features like data redaction, and monitor usage for cost and compliance. If the chosen AI model for invoice processing proves too expensive, the gateway facilitates an easy switch to a more cost-effective alternative without disrupting the core business process. * Supply Chain Optimization: AI models can predict demand fluctuations, optimize logistics routes, or identify potential disruptions. The AI Gateway provides the necessary infrastructure to feed real-time supply chain data to these models, abstracting the complexity of diverse AI APIs and providing crucial observability into the performance and cost of these AI-driven optimizations.

Optimizing MLOps

Machine Learning Operations (MLOps) encompasses the entire lifecycle of ML models, from development to deployment and monitoring. The Cloudflare AI Gateway significantly streamlines several MLOps aspects. * Streamlined Model Deployment: Instead of needing to adapt application code every time a new version of an ML model is deployed or a new model is introduced, MLOps teams can simply update the routing configuration in the AI Gateway. This decouples application development from model deployment, accelerating iteration cycles. * A/B Testing and Canary Deployments: The gateway can be configured to route a small percentage of traffic to a new model version (canary deployment) or to test multiple models simultaneously (A/B testing). This allows MLOps teams to compare performance, cost, and output quality of different models in a production environment before a full rollout, all managed at the edge without impacting the core application. * Consistent Monitoring and Alerting: By centralizing all AI requests, the gateway offers a single pane of glass for monitoring model performance, latency, and error rates. This provides critical data for MLOps teams to quickly identify and address issues, ensuring model reliability and performance.

Ensuring Data Privacy and Compliance

For organizations dealing with sensitive information, data privacy and regulatory compliance are non-negotiable. The AI Gateway offers powerful tools to mitigate risks. * GDPR, CCPA, and HIPAA Compliance: Features like prompt logging control, data masking, and granular access controls are crucial for adhering to stringent data protection regulations. The ability to redact or anonymize personally identifiable information (PII) before it reaches an external AI model is a game-changer for industries like healthcare and finance. * Preventing Data Leakage: By centralizing and securing AI interactions, the gateway reduces the risk of sensitive data accidentally being exposed to unintended AI models or stored in unsecure logs. All data flowing to and from AI models is subject to the gateway's security policies.

Developing Multi-AI Model Strategies

The AI landscape is too dynamic and specialized for a one-size-fits-all approach. Businesses increasingly need to leverage multiple AI models for different tasks, or even for the same task with different characteristics. * Fallback Models: If a primary AI model becomes unavailable or returns an error, the AI Gateway can automatically route the request to a fallback model, ensuring continuous service availability. This drastically improves the resilience of AI-powered applications. * Tiered Model Access: Organizations can implement strategies where simpler or less critical requests are handled by cheaper, faster models (e.g., smaller LLMs), while more complex or critical requests are routed to more powerful, potentially more expensive models. This optimizes both performance and cost. * Semantic Routing: Advanced configurations could involve routing requests based on their semantic content. For example, a legal query might go to a specialized legal LLM, while a creative writing prompt goes to a generative text model, all transparently managed by the gateway.

In all these scenarios, the Cloudflare AI Gateway acts as a strategic enabler, transforming the chaotic world of AI integration into a structured, secure, and highly performant operational domain. It empowers developers and enterprises alike to focus on extracting business value from AI, rather than getting entangled in its underlying complexities.

APIPark is a high-performance AI gateway that allows you to securely access the most comprehensive LLM APIs globally on the APIPark platform, including OpenAI, Anthropic, Mistral, Llama2, Google Gemini, and more.Try APIPark now! 👇👇👇

Technical Implementation Details

Implementing the Cloudflare AI Gateway involves a series of straightforward steps that leverage Cloudflare's existing infrastructure and configuration paradigms. While the full deployment guide would reside in Cloudflare's official documentation, understanding the conceptual flow and key configuration points provides a clear picture of its technical utility.

Setting Up Cloudflare AI Gateway (Conceptual Steps)

The initial setup of the Cloudflare AI Gateway primarily involves configuring it within your Cloudflare dashboard and then directing your application's AI requests to the new gateway endpoint.

1. Cloudflare Account and Domain: You'll need an active Cloudflare account and a domain managed by Cloudflare. The AI Gateway functionality is typically accessible within the Cloudflare dashboard, often under a dedicated AI or Workers AI section.
2. Creating a Gateway: Within the Cloudflare dashboard, you'll initiate the creation of a new AI Gateway. This involves defining a unique name for your gateway and specifying the target AI models it will interact with. For instance, you might configure it to proxy requests to api.openai.com for specific OpenAI models, or to a custom endpoint for a self-hosted model.
3. Configuring Upstream AI Services: For each AI model you intend to use (e.g., gpt-3.5-turbo, claude-2), you will configure its specific details. This includes the upstream API endpoint (e.g., https://api.openai.com/v1/chat/completions), the required API keys or authentication headers (which the gateway securely manages), and any specific model identifiers. The gateway handles the secure storage and injection of these credentials into outgoing requests, preventing your application from needing to directly manage sensitive API keys.
4. Gateway Endpoint Generation: Upon configuration, the Cloudflare AI Gateway will provide you with a unique, secure endpoint (e.g., https://your-domain.cloudflareai.com/v1/chat/completions). This is the URL your applications will use to make all AI requests, abstracting away the actual backend AI provider.
5. Updating Application Code: Your client applications, instead of calling api.openai.com directly, will now send their requests to your newly created Cloudflare AI Gateway endpoint. The request payload (e.g., JSON for chat completions) generally remains the same, as the gateway intelligently translates and forwards it. This minimal code change makes adoption relatively simple.

Configuration Options for Caching, Rate Limits, and Logging

Once the gateway is set up, you can fine-tune its behavior through various configuration options to optimize performance, cost, and security.

• Caching Rules:
  • Cache Duration (TTL): Define how long responses from AI models should be stored in the cache before being considered stale. For rapidly changing data or highly dynamic AI responses, a short TTL is appropriate. For static or frequently repeated queries (e.g., common FAQ answers from an LLM), a longer TTL can yield significant cost savings and performance benefits.
  • Cache Keys: Specify which parts of the incoming request (e.g., prompt content, model name, user ID) should be used to generate a unique cache key. This ensures that only identical requests retrieve cached responses.
  • Bypass Cache: Configure rules to explicitly bypass the cache for certain requests, for example, for critical real-time interactions that always require the freshest AI response.
• Rate Limiting Policies:
  • Limit per Time Unit: Define the maximum number of requests allowed within a specific time frame (e.g., 100 requests per minute).
  • Burst Limit: Allow a temporary spike in requests above the steady-state limit for a short period before enforcement, accommodating natural traffic fluctuations.
  • Based on Criteria: Apply rate limits based on various request attributes, such as IP address, API key, user ID, or specific AI model being invoked. This allows for granular control and preventing individual bad actors from monopolizing resources.
  • Action on Exceedance: Specify the action to take when a rate limit is exceeded, typically returning an HTTP 429 "Too Many Requests" status code.
• Logging and Observability Settings:
  • Log Retention: Configure how long logs of AI requests and responses are retained within the Cloudflare platform. This is crucial for compliance and auditing.
  • Prompt/Response Redaction: For sensitive data, implement rules to automatically redact specific patterns (e.g., credit card numbers, PII) from prompts before logging or even before sending to the AI model. This is a powerful privacy control.
  • Sampling: For high-volume traffic, you might configure logging to only sample a percentage of requests to manage storage costs while still gaining representative insights.
  • Integration with SIEM/Log Management: Cloudflare AI Gateway logs can often be integrated with external Security Information and Event Management (SIEM) systems or centralized log management platforms for comprehensive enterprise-wide visibility and correlation.

API Keys and Authentication Mechanisms

Security is paramount when interacting with AI models. The Cloudflare AI Gateway provides robust mechanisms for managing API keys and securing access. * Secure API Key Storage: Instead of hardcoding AI provider API keys in your application, you provide them securely to the Cloudflare AI Gateway. The gateway stores these keys encrypted and injects them into outgoing requests to the backend AI service. This significantly reduces the risk of API key compromise if your application's client-side code is exposed. * Client Authentication: The gateway allows you to authenticate your client applications using various methods: * Cloudflare API Tokens: Use Cloudflare-specific API tokens to authenticate your application with the AI Gateway itself. * JWT (JSON Web Tokens): Integrate with your existing identity providers using JWTs, allowing for fine-grained authorization policies based on user roles or permissions embedded in the token. * Cloudflare Access: For internal applications, Cloudflare Access can secure the AI Gateway endpoint, ensuring that only authorized users or devices from your organization can access it. * Fine-grained Authorization: Beyond simple authentication, the gateway can enforce authorization rules, allowing certain applications or users to access only specific AI models or perform particular types of requests.

Integration with Cloudflare Workers and Pages for Custom Logic

One of the most powerful aspects of the Cloudflare AI Gateway is its seamless integration with Cloudflare Workers. Workers are serverless functions that run on Cloudflare's edge network, allowing developers to execute custom code incredibly close to the user. * Pre-processing and Post-processing: * Request Transformation: Before forwarding a request to an AI model, a Worker can modify the prompt, add context, or standardize the input format. For example, ensuring all prompts adhere to a specific schema. * Response Modification: After receiving a response from an AI model, a Worker can process it—e.g., format the output, add branding, or perform further data redaction before sending it back to the client. * Dynamic Model Selection: A Worker can implement custom logic to dynamically choose which AI model to use based on the content of the prompt, user characteristics, or current model performance metrics. This allows for highly intelligent routing beyond simple load balancing. * Complex Authorization Rules: Workers can implement more sophisticated authorization logic, such as checking against an external database of user permissions or integrating with custom identity systems. * Advanced Caching Strategies: While the AI Gateway offers built-in caching, Workers can implement more nuanced caching logic, such as cache invalidation based on specific events or multi-level caching strategies. * Hybrid AI Deployments: Cloudflare Workers can also directly interact with AI models running on Cloudflare's own Workers AI platform, which enables running inference on open-source models directly at the edge, offering even lower latency for certain tasks and a complementary approach to the proxying capabilities of the AI Gateway. * APIPark Integration Considerations: While Cloudflare Workers are excellent for augmenting Cloudflare's own services, for organizations seeking a self-hosted, open-source platform with full API lifecycle management and quick integration of 100+ AI models, APIPark provides a robust alternative. Its strength lies in offering a unified API format for AI invocation, prompt encapsulation into REST APIs, and powerful data analysis, all within an Apache 2.0 licensed framework. Businesses can leverage APIPark's comprehensive features for managing not only AI models but also traditional REST services, providing an end-to-end API governance solution that rivals Nginx in performance and offers detailed logging and analytics, suitable for those who prioritize control and an open-source ecosystem. This offers a different deployment and management philosophy compared to Cloudflare's fully managed, edge-centric service, appealing to distinct enterprise needs. You can learn more about its capabilities at ApiPark.

Monitoring Dashboards and Alerts

Cloudflare provides comprehensive dashboards and alerting capabilities for the AI Gateway, ensuring operational transparency. * Real-time Dashboards: Access intuitive dashboards that display key metrics like total requests, cached requests, cache hit ratio, latency distribution, error rates, and token usage for all your AI interactions. These dashboards allow for quick health checks and performance monitoring. * Customizable Alerts: Configure alerts based on specific thresholds for these metrics. For example, receive notifications if the error rate for an AI model exceeds a certain percentage, if latency spikes, or if token usage approaches a predefined budget. These proactive alerts enable rapid response to potential issues before they impact users or incur significant costs. * Audit Logs: Detailed audit logs track all configuration changes made to the AI Gateway, providing accountability and ensuring compliance.

By providing these granular technical controls and deep integration possibilities, the Cloudflare AI Gateway empowers developers and operations teams to deploy, manage, and optimize their AI workloads with precision, security, and efficiency, all while leveraging the power of Cloudflare's global edge network.

Cloudflare AI Gateway vs. Traditional API Gateways vs. Custom Solutions

Understanding the distinct advantages of the Cloudflare AI Gateway requires a comparative lens, examining how it differs from traditional API Gateway solutions and custom-built proxy infrastructures. While all aim to mediate API traffic, their approaches and capabilities for AI workloads vary significantly.

Traditional API Gateways

A traditional API Gateway has been a staple in modern microservices architectures for years. Its primary functions include: * Request Routing: Directing incoming requests to the correct backend service. * Authentication & Authorization: Verifying client identity and permissions. * Rate Limiting: Protecting backend services from overload. * Load Balancing: Distributing traffic across multiple instances of a service. * Protocol Translation: Sometimes converting between different protocols (e.g., HTTP to gRPC). * API Management: Providing features like developer portals, documentation, and versioning.

Where they do well: For managing standard RESTful APIs and microservices, traditional gateways are highly effective. They centralize common concerns, enforce policies, and provide a single entry point for API consumers. Products like Nginx, Kong, Apigee, or Amazon API Gateway excel in this domain.

Where they fall short for AI: * AI-Specific Caching: Traditional gateways often have generic caching mechanisms, but they typically lack intelligent, AI-aware caching tailored for LLM responses (e.g., token-based billing awareness, or prompt-based cache key generation). * Prompt Engineering & Versioning: They don't inherently understand or manage the nuances of prompt engineering, prompt transformation, or versioning of prompts, which is critical for consistent and optimized AI interactions. * AI-Specific Security: While offering general security, they lack built-in features for prompt injection detection, sensitive data redaction before sending to AI models, or logging controls specifically designed for AI's unique privacy concerns. * AI Model Agnosticism: Integrating a new AI model with a traditional gateway often means configuring a new upstream endpoint and potentially writing custom code or policies for each model's specific API quirks. They don't provide a unified abstraction layer over diverse AI providers out-of-the-box. * Cost Management for AI: Generic API call tracking doesn't translate directly into AI-specific costs (e.g., per token billing for LLMs), making cost optimization and tracking challenging. * Edge Performance: While some can be deployed globally, their core design isn't always as intrinsically tied to an extensive, low-latency edge network as Cloudflare's.

Cloudflare AI Gateway's Unique Advantages

The Cloudflare AI Gateway is purpose-built to overcome these limitations, offering a specialized and optimized solution for AI workloads. * Native AI-Centric Features: It is designed from the ground up with AI interactions in mind. Features like intelligent prompt caching (aware of token costs), prompt logging controls for privacy, and dynamic model routing are integrated natively. This makes it a true LLM Gateway. * Global Edge Network Integration: Leveraging Cloudflare's vast global network, requests are processed at the edge, closest to the user. This dramatically reduces latency for AI inferences and improves user experience, especially for interactive AI applications. Traditional gateways might offer global deployment, but Cloudflare's network is an inherent part of its architecture, not an add-on. * Integrated Security: It benefits from Cloudflare's full suite of integrated security services, including WAF (which can be tailored for AI-specific attacks), DDoS protection, and bot management, all applied at the edge. This provides a multi-layered defense specifically adapted for the unique threat landscape of AI. * Unified Abstraction Layer: It acts as a single pane of glass for diverse AI models and providers. Applications interact with a consistent API, and the gateway handles the underlying complexity of different AI APIs, allowing for seamless model swapping and A/B testing. * Granular AI Cost Management: Provides deep visibility into AI usage metrics like token consumption, which directly correlates to AI billing, enabling more accurate cost tracking, allocation, and optimization strategies. * Developer Productivity: By abstracting away AI complexities, developers can focus on building innovative applications rather than wrestling with integration details, API changes, or security nuances of multiple AI providers.

Custom-Built Solutions

Some organizations might consider building their own custom AI Gateway or proxy solution using open-source tools or internal engineering resources. * Potential Advantages: * Full Control: Complete control over every aspect of the solution, tailoring it precisely to specific needs. * No Vendor Lock-in (for the gateway itself): Not dependent on a third-party managed service for the gateway infrastructure. * Significant Disadvantages: * High Development Cost: Building and maintaining a production-grade AI gateway with features like caching, rate limiting, security, observability, and scalability is an enormous engineering undertaking, requiring significant upfront investment in time, money, and specialized talent. * Maintenance Overhead: Ongoing maintenance, patching, updates, and debugging of a custom solution create a substantial operational burden. * Lack of Scalability & Resilience: Achieving global scalability, high availability, and resilience on par with Cloudflare's network is extremely difficult and costly for an individual organization. * Security Gaps: Ensuring comprehensive security against evolving threats, including AI-specific vulnerabilities, requires constant vigilance and expertise that many internal teams may lack. * Feature Parity: Matching the feature set (e.g., prompt management, advanced caching, detailed analytics) of a specialized AI Gateway like Cloudflare's is challenging.

APIPark: An Open-Source Alternative for Comprehensive API Management

In the landscape of AI Gateway solutions, it's also important to acknowledge alternatives that cater to different needs and deployment preferences. For organizations seeking a self-hosted, open-source solution that provides not only a powerful AI Gateway but also a comprehensive API Management Platform, APIPark stands out. Released under the Apache 2.0 license, APIPark offers a compelling suite of features that address many of the same challenges as Cloudflare's offering, but with a distinct focus on giving enterprises full control over their API infrastructure.

APIPark integrates a variety of AI models (supporting over 100+) with a unified management system for authentication and cost tracking, similar to what a robust LLM Gateway should provide. Its core strength lies in standardizing the request data format across all AI models, ensuring that application-level changes are minimized even if the underlying AI models or prompts evolve. This significantly simplifies AI usage and reduces maintenance costs. Furthermore, APIPark empowers users to encapsulate custom prompts with AI models to quickly create new, specialized REST APIs (e.g., for sentiment analysis or translation), offering a powerful way to productize AI capabilities.

Beyond AI-specific features, APIPark excels as an end-to-end API lifecycle management platform. It assists with design, publication, invocation, and decommissioning of both AI and traditional REST APIs, regulating management processes, and handling traffic forwarding, load balancing, and versioning. For teams, it offers centralized API service sharing and supports multi-tenancy with independent API and access permissions for each tenant, enhancing resource utilization and security. Its performance rivals Nginx, capable of over 20,000 TPS on modest hardware, and it provides detailed API call logging and powerful data analysis for troubleshooting and preventive maintenance.

APIPark is an excellent choice for businesses that prioritize:

• Open Source Control: Desire for an Apache 2.0 licensed solution with the ability to self-host and customize.
• Comprehensive API Lifecycle Management: Need a single platform for managing both AI and traditional REST APIs from design to decommission.
• Unified AI Model Integration: Seamlessly integrate and manage a vast array of AI models through a standardized API.
• Prompt as API: The ability to easily turn custom prompts into reusable REST APIs.
• High Performance and Scalability: A robust gateway that can handle large-scale traffic with Nginx-like performance.
• Detailed Observability for Self-Hosted: In-depth logging and data analysis tools for self-managed deployments.

While Cloudflare AI Gateway focuses on leveraging a global edge network and fully managed services, APIPark offers a powerful, open-source, and self-hosted alternative that gives businesses granular control and comprehensive API management capabilities across their entire API ecosystem. For more information, visit ApiPark.

The choice between these solutions ultimately depends on an organization's specific needs, existing infrastructure, budget, and strategic priorities regarding control, deployment model, and the extent of API management desired.

The Future of AI and Gateways

The trajectory of Artificial Intelligence is one of relentless innovation and increasing integration into every facet of our digital lives. As AI models become more sophisticated, specialized, and pervasive, the role of intelligent AI Gateway solutions will only grow in importance. The future landscape will likely be shaped by several key trends, each underscoring the necessity of robust LLM Gateway and broader AI Gateway infrastructures.

Evolving AI Landscape: Multimodal Models, Smaller Specialized Models

The AI world is rapidly diversifying. While LLMs currently dominate headlines, the next wave is already here: * Multimodal AI: Models capable of understanding and generating content across multiple modalities—text, image, audio, video—are becoming more prevalent. These models introduce new complexities in input/output formats and processing requirements. An AI Gateway will need to evolve to natively support these multimodal interactions, orchestrating requests to different components of a multimodal model or even translating between modalities on the fly. * Smaller, Specialized Models: Alongside giant foundation models, we're seeing a rise in smaller, fine-tuned, and highly specialized models designed for specific tasks (e.g., medical diagnosis, legal analysis, code generation for a niche language). These models offer efficiency and precision but also multiply the number of different AI services an organization might need to manage. The AI Gateway will become crucial for routing requests to the most appropriate specialized model, optimizing for both accuracy and cost. This proliferation will further solidify the need for a unified LLM Gateway that can seamlessly switch between model sizes and specializations. * Real-time AI: The demand for instantaneous AI responses (e.g., for live translation, real-time gaming AI, autonomous driving) will push the boundaries of latency. Gateways will need even more advanced caching and edge processing capabilities to meet these ultra-low latency requirements.

Increased Need for Robust LLM Gateway Solutions

Large Language Models will continue to be central to many AI applications, and the challenges associated with them will necessitate even more powerful LLM Gateway features: * Advanced Prompt Management: As prompt engineering becomes more sophisticated, LLM gateways will likely offer more robust tools for prompt versioning, A/B testing, dynamic prompt injection based on user context, and even AI-driven prompt optimization. * Guardrails and Responsible AI: The ethical implications of LLMs are a growing concern. Future LLM gateways will incorporate more sophisticated guardrails to prevent harmful output, detect biases, and ensure compliance with emerging AI regulations. This might involve integrating with specialized safety models or applying content moderation at the gateway level. * Cost Optimization for Diverse Billing Models: LLM billing models are evolving (e.g., per-token, per-call, per-compute-hour, context window size). An intelligent LLM Gateway will need to adapt its cost tracking and optimization strategies to these diverse models, providing more accurate spend prediction and control. * Stateful Interactions: As LLMs move towards longer-form, more conversational interactions, gateways may need to manage session state and context more effectively, perhaps by intelligently summarizing past conversations before sending subsequent prompts to the LLM to manage token limits and maintain coherence.

The Role of Edge Computing in AI Inference

Cloudflare's strategy of deploying its AI Gateway at the edge is not just a present advantage but a foundational element for the future of AI. * Lower Latency: As AI becomes more interactive and real-time, performing inference as close to the user as possible (the "edge") is paramount. Edge-based AI Gateways reduce the round-trip time to origin AI models, and increasingly, will even host smaller AI models for direct inference at the edge. * Data Locality and Privacy: For sensitive data, performing inference at the edge can help keep data within specific geographical boundaries or even within an organization's controlled network segment, addressing privacy and compliance requirements. This minimizes data movement and potential exposure. * Reduced Bandwidth Costs: By performing some inference or caching responses at the edge, organizations can significantly reduce the amount of data that needs to travel back and forth to centralized AI data centers, lowering bandwidth costs. * Offline Capabilities: In the long term, edge computing could enable limited AI functionality even when connectivity to centralized cloud resources is intermittent or unavailable, supporting applications in remote or disconnected environments.

Ethical Considerations and Governance in AI

As AI's influence expands, ethical considerations and robust governance frameworks become critical. * Transparency and Explainability: Gateways might play a role in logging and exposing the lineage of AI outputs, helping to explain why a particular response was generated or which models were involved. * Compliance and Regulation: As governments introduce AI-specific regulations (e.g., EU AI Act), AI Gateways will be essential for enforcing these policies, ensuring data provenance, and auditing AI interactions for compliance. * Bias Detection and Mitigation: Future gateways might integrate with tools that monitor AI outputs for biases, providing warnings or even modifying responses to promote fairness.

The future of AI is undeniably exciting, promising unparalleled innovation and efficiency. However, realizing this potential safely, efficiently, and ethically hinges on the continued evolution of supporting infrastructure like the Cloudflare AI Gateway. These intelligent intermediaries will not only abstract complexity but will also serve as critical control points for security, cost, performance, and responsible AI governance, ensuring that the AI revolution is both powerful and manageable.

Conclusion

The journey into the age of Artificial Intelligence is marked by boundless potential, yet it is also paved with intricate technical, security, and operational challenges. From the fragmented landscape of diverse AI models and APIs to the critical demands of data privacy, cost control, and performance at scale, organizations face a complex integration puzzle. The Cloudflare AI Gateway emerges not merely as a convenient tool but as an indispensable strategic asset, providing a sophisticated, unified, and secure solution to navigate this complexity.

We have explored how the Cloudflare AI Gateway, leveraging Cloudflare's expansive global edge network, transforms the chaotic world of AI integration into a streamlined and manageable process. By serving as an intelligent AI Gateway and a specialized LLM Gateway, it unifies disparate AI APIs, enabling seamless interaction with a multitude of models from various providers. Its robust caching mechanisms slash latency and dramatically reduce operational costs, while comprehensive rate limiting and load balancing ensure stability and fair usage. Security is paramount, with features designed specifically for AI, including prompt logging controls, data redaction, and integrated WAF protection against emerging threats like prompt injection. Moreover, the deep observability and analytics it provides offer unparalleled insight into AI consumption, performance, and expenditure, empowering businesses to make informed decisions and optimize their AI strategies.

By abstracting away the underlying complexities, Cloudflare AI Gateway empowers developers to accelerate the creation of innovative AI-powered applications, allowing them to focus on unique value proposition rather than infrastructure headaches. For enterprises, it facilitates secure, compliant, and scalable AI adoption across existing workflows, reducing risk and accelerating return on AI investments. While alternative solutions like APIPark offer a robust open-source, self-hosted option for comprehensive API lifecycle management with strong AI integration capabilities, Cloudflare's fully managed, edge-centric approach provides a distinct advantage for those prioritizing global distribution, integrated security, and minimal operational overhead.

In essence, the Cloudflare AI Gateway democratizes access to cutting-edge AI, making it more accessible, more secure, and more cost-effective for everyone. It is the critical infrastructure layer that bridges the gap between raw AI potential and practical, scalable, and secure deployment. As AI continues its rapid evolution, an intelligent gateway will remain central to unlocking its true transformative power, ensuring that businesses can harness this revolutionary technology with confidence and control, paving the way for an innovative, AI-powered future.

---

Frequently Asked Questions (FAQs)

1. What exactly is an AI Gateway and how does it differ from a traditional API Gateway? An AI Gateway is a specialized form of an API Gateway designed specifically for managing interactions with Artificial Intelligence models. While a traditional API Gateway handles general API traffic (routing, authentication, rate limiting for REST APIs), an AI Gateway adds AI-specific features like intelligent prompt caching (aware of token costs), prompt management, AI-specific security (e.g., prompt injection detection, data redaction before sending to models), and unified abstraction over diverse AI model APIs (e.g., from OpenAI, Google, Hugging Face). It's built to address the unique complexities, costs, and security concerns of AI workloads, especially Large Language Models (LLMs), making it a dedicated LLM Gateway.

2. How does Cloudflare AI Gateway help with managing AI costs? Cloudflare AI Gateway provides several mechanisms for cost management. Primarily, its intelligent caching dramatically reduces the number of redundant calls to expensive AI models (like LLMs), directly cutting down token-based or per-call costs. It also offers detailed observability and analytics, allowing organizations to track AI usage per model, application, or user, identify spending patterns, and set budget alerts. This granular visibility empowers teams to optimize their AI strategies and avoid unexpected expenses.

3. What security benefits does Cloudflare AI Gateway offer for AI applications? The Cloudflare AI Gateway provides robust, AI-specific security features. It allows for secure storage and management of AI provider API keys, preventing their exposure in client applications. Crucially, it offers prompt logging controls and data redaction capabilities, allowing businesses to mask or remove sensitive information from prompts before they reach external AI models and before they are logged, ensuring data privacy and compliance. Additionally, it integrates with Cloudflare's Web Application Firewall (WAF) and DDoS protection, safeguarding against general web attacks and AI-specific vulnerabilities like prompt injection, all applied at the network edge.

4. Can Cloudflare AI Gateway work with multiple AI models and providers simultaneously? Yes, absolutely. One of the core strengths of the Cloudflare AI Gateway is its ability to provide a unified API abstraction layer over a multitude of AI models from various providers (e.g., OpenAI, Google, Hugging Face). Your applications interact with a single, consistent gateway endpoint, and the gateway handles the routing, authentication, and transformation required for each specific backend AI model. This allows for seamless switching between models, A/B testing, and building resilient applications with fallback models, all without requiring changes to your core application code.

5. How does Cloudflare's edge network enhance the performance of AI Gateway? Cloudflare's extensive global edge network, with data centers in hundreds of cities worldwide, places the AI Gateway physically closer to end-users. This geographical proximity significantly reduces latency, as AI requests and responses travel shorter distances, resulting in faster processing and a more responsive user experience for AI-powered applications. Furthermore, performing caching and other gateway functions at the edge offloads traffic from origin AI models, enhancing overall system reliability and performance by minimizing the load on backend services and reducing bandwidth costs.

🚀You can securely and efficiently call the OpenAI API on APIPark in just two steps:

Step 1: Deploy the APIPark AI gateway in 5 minutes.

APIPark is developed based on Golang, offering strong product performance and low development and maintenance costs. You can deploy APIPark with a single command line.

curl -sSO https://download.apipark.com/install/quick-start.sh; bash quick-start.sh

In my experience, you can see the successful deployment interface within 5 to 10 minutes. Then, you can log in to APIPark using your account.

Step 2: Call the OpenAI API.