Can I serve a LoRA adapter I trained?

Yes. Hand it to While and we import it. Or import it into Bedrock yourself and register the ARN. Any OpenAI-compatible server you run registers by URL.

Does the served model score the same as the trained one?

Yes on the adapter we checked. It solved 35 of 100 held-out tasks through Bedrock. It solved 35 of 100 on vLLM. The difference is half a point and could be chance.

Can I use my own domain?

You get a subdomain on models.withwhile.com. Claim it once. For your own domain write to us.

Hosting

Host your model.

You trained it. Put it behind an OpenAI-compatible endpoint on While. and get a subdomain of your own.

One URL for every model

Runs where it runs

A Bedrock import in your AWS account or ours. Or any /v1 server you host. We store no key of yours.

Your subdomain

Claim acme.models.withwhile.com in one call. Only your keys work there.

For a model in your own AWS account add a role named WhileModelsInvoke that trusts While. and can invoke the model. The endpoint assumes it on every call.

curl -X POST https://models.withwhile.com/models \
  -H "Authorization: Bearer $WHILEAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"name": "nemotron-8b-t2s-r1",
       "arn": "arn:aws:bedrock:us-east-1:<account>:imported-model/<id>",
       "roleArn": "arn:aws:iam::<account>:role/WhileModelsInvoke"}'

Call it

Use the OpenAI SDK or curl or wai.Endpoint.

from openai import OpenAI

client = OpenAI(base_url="https://models.withwhile.com/v1", api_key="zp_...")
reply = client.chat.completions.create(
    model="nemotron-8b-t2s-r1",
    messages=[{"role": "user", "content": "Which customers never ordered?"}],
    stream=True,
)

Your subdomain

curl -X PUT https://models.withwhile.com/domain \
  -H "Authorization: Bearer $WHILEAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"subdomain": "acme"}'
# https://acme.models.withwhile.com/v1 now answers for your account only

The served weights are the trained weights

We imported a text-to-SQL adapter into Bedrock. We scored it on the same 140 held-out tasks as the vLLM run.

Through the endpoint: 35 of 100 tasks
On vLLM: 35 of 100 tasks
The base model: 26 of 100 tasks

The two servings are half a point apart. That could be chance. Both beat the base by 8 points. That is not chance. The recipe.

What we keep

Never a prompt. Never a completion. Never a request log.
Per model per day we count calls and errors and tokens and the five-minute windows it answered in. That is the bill.
Each account gets 600 requests a minute. Every call is over TLS behind a firewall.

What it costs

Three prices when While. hosts the model. A unit is a Bedrock Custom Model Unit. An 8B model is 2 units.

While it answers: $0.12; per unit-minute in five-minute windows
While it is kept: $5; per unit per month
Per import: $10; one time

Your own AWS account or server is free. An idle model wakes in about two minutes. The OpenAI SDKs wait for it. Worked months are on the pricing page.

Questions

Can I serve a LoRA adapter I trained?: Yes. Hand it to While. and we import it. Or import it into Bedrock yourself and register the ARN. Any OpenAI-compatible server you run registers by URL.
Does the served model score the same as the trained one?: Yes on the adapter we checked. It solved 35 of 100 held-out tasks through Bedrock. It solved 35 of 100 on vLLM. The difference is half a point and could be chance.
Can I use my own domain?: You get a subdomain on models.withwhile.com. Claim it once. For your own domain write to us.

Read the docs Ask about your own domain Home