Alice

Evaluate prompts and model responses against the Alice policies configured for your application — block, mask, or record a detection.

Alice evaluates prompts and model responses against the policies you configure per application, and returns a verdict Portkey enforces: allow it, block it, mask the flagged spans before the model sees them, or record a detection and let it through.

 

Using Alice with Portkey

1. Add Alice Credentials to Portkey

• Navigate to the Integrations page under Sidebar
• Click on the edit button for the Alice integration
• Add your Alice API key — create one in WonderSuite under Account Settings → API Keys

API base URL is optional. Leave it blank unless you run a private Alice deployment.

2. Add Alice’s Guardrail Check

• Navigate to the Guardrails page and click the Create button
• Search for Evaluate and click Add
• Set the Application the check applies to (see below), and any actions you want on your check
• Create the Guardrail!

Guardrail Actions allow you to orchestrate your guardrails logic. You can learn them here

Check Name	Description	Parameters	Supported Hooks
Evaluate	Evaluate the prompt or the model response against the Alice policies configured for your application, and enforce the verdict.	Application, Timeout	beforeRequestHook, afterRequestHook

Parameters:

Parameter	Type	Default	Description
appId	string	—	Required. The application whose policies apply
timeout	number	5000	Timeout in milliseconds

Naming the application

This is the one setting most likely to trip you up on a first run, so it is worth a section of its own.

Alice configures policies per application, and a project usually holds several — a support bot and a payments agent do not want the same rules. One Alice API key fronts all of them, so the application cannot be part of the credential. It is a parameter on the check instead:

{
  "id": "alice.evaluate",
  "parameters": { "appId": "payments-bot" }
}

appId takes the application’s UUID from your Alice Application Inventory, or your own id for it if you filled in the optional Application ID field when you added the application. A guardrail that names no application fails its check rather than being evaluated against a guess — which would measure your traffic against the wrong policy set.

If you need different policies for different traffic, create one guardrail per application and attach each to the Config that routes that traffic.

The application is deliberately not read from request metadata. x-portkey-metadata is written by the caller, so an application named there would let a caller point their own traffic at policies laxer than the ones you configured for them. Guardrail configuration is the only place it is read from.

What is sent

The plugin forwards the hook context as it received it and enforces the verdict that comes back. It selects nothing and renames nothing — which parts of a turn are worth evaluating, and how a verdict is reached, are decided on Alice’s side, so those can change without you upgrading your Gateway.

Credentials are the single exception. context.request.headers carries your caller’s Authorization and x-portkey-api-key in the clear, and a provider credential can ride along under a nested field, so any key named headers, credentials, api_key, apiKey, authorization or cookie is dropped at every nesting depth before the body is serialised, and never leaves your Gateway. Only the outbound copy is affected — the real context continues down the pipeline untouched.

Portkey’s x-portkey-trace-id is sent alongside, so the turns of one conversation group together in Alice rather than scattering.

What each verdict does

Verdict	Portkey does
ALLOW	The request proceeds
MASK	The flagged spans are replaced before the model sees them, and the request proceeds
BLOCK	The check fails, carrying your policy’s own message
DETECT	The request proceeds; the explanation carries the correlation id of Alice’s record

A MASK verdict is applied all-or-nothing: if any replacement cannot be written back, the check fails instead of applying the rest, so content Alice meant to replace can never reach the model alongside content that was replaced.

When Alice cannot be reached

A transport failure, a timeout, and an answer that cannot be read are all reported as a check error, never as a pass. So is a 4xx — a rejected API key is a misconfiguration, not a reason to let traffic through unscreened.

That makes failOnError on the check the single place that decides what happens next:

• failOnError: false (default) — the guardrail is skipped and the request proceeds
• failOnError: true — the check fails, and your guardrail actions decide whether to deny

Set it to true if unscreened traffic is not acceptable for this application.

Masking applies to the content part currently being handled. On a streamed response, blocking still works, but the streamed text is not rewritten.

3. Add Guardrail ID to a Config and Make Your Request

• When you save a Guardrail, you’ll get an associated Guardrail ID - add this ID to the input_guardrails or output_guardrails params in your Portkey Config
• Create these Configs in Portkey UI, save them, and get an associated Config ID to attach to your requests. More here.

Here’s an example config:

{
  "input_guardrails": ["guardrails-id-xxx"],
  "output_guardrails": ["guardrails-id-xxx"]
}

NodeJS

const portkey = new Portkey({
    apiKey: "PORTKEY_API_KEY",
    config: "pc-***" // Supports a string config id or a config object
});

Python

portkey = Portkey(
    api_key="PORTKEY_API_KEY",
    config="pc-***" # Supports a string config id or a config object
)

OpenAI NodeJS

const openai = new OpenAI({
  apiKey: 'OPENAI_API_KEY',
  baseURL: PORTKEY_GATEWAY_URL,
  defaultHeaders: createHeaders({
    apiKey: "PORTKEY_API_KEY",
    config: "CONFIG_ID"
  })
});

OpenAI Python

client = OpenAI(
    api_key="OPENAI_API_KEY",
    base_url=PORTKEY_GATEWAY_URL,
    default_headers=createHeaders(
        provider="openai",
        api_key="PORTKEY_API_KEY",
        config="CONFIG_ID"
    )
)

REST

curl https://api.portkey.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "x-portkey-api-key: $PORTKEY_API_KEY" \
  -H "x-portkey-config: $CONFIG_ID" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [{
        "role": "user",
        "content": "Hello!"
      }]
  }'

For more, refer to the Config documentation.

Your requests are now guarded by Alice and you can see the Verdict and any action you take directly on Portkey logs!

Get Support

If you face any issues with the Alice integration, see the Alice docs or ping the Portkey team on the community forum.