How to Lock Down an SGLang or vLLM Inference Server (2026)

This guide is for anyone running SGLang or vLLM on their own GPUs who read about NEAR AI's breach through an unpatched SGLang RCE and wants to know what to change today. At the end your engine will listen only on loopback or a pod network, inference and admin calls will need different keys, a reverse proxy will pass only the routes you use, and multi-node traffic will be fenced off. There is no upstream patch for the NEAR flaw yet, so these are the controls that limit damage while you wait. Versions: SGLang v0.5.23 (on PyPI since 30 October 2026), vLLM 0.33.0, nginx 1.27. You need shell access to the engine host or the Kubernetes manifests, and a TLS certificate for the proxy.

What I actually ran for this guide: I downloaded the SGLang 0.5.23 wheel, read its HTTP server and auth code, and ran its auth decision function against every key setup (Step 2). I ran the nginx config from Step 3 locally against a stub upstream and recorded which paths got through. I did not run SGLang or vLLM on a GPU for this guide, and I did not apply the NetworkPolicy to a cluster.

Why inference engines are an easy target

Inference engines are built for researchers on trusted clusters. They ship management endpoints for loading weights, swapping LoRA adapters, profiling and flushing caches, and several of those deserialize data. CERT/CC's VU#281278 from July 2026 lists six SGLang CVEs, including code execution through /load_lora_adapter_from_tensors and /update_weights_from_disk via unsafe pickle handling, an SSRF through image_url, and API keys leaking from /server_info. CERT said no patches were available when it published. vLLM has its own run: GHSA-h3rc-j82p-2c3h, an 8.1 code execution chain through untrusted EPS images, was fixed in 0.32.0 on 20 October.

The pattern is the same each time: an endpoint that should never face a client is reachable by one. So the work is mostly about reachability.

Peer engine pods NCCL, rendezvous ports With only --api-key set, SGLang 0.5.23 accepts that key on weight and LoRA loading routes. Set both keys, give clients only the API key, and keep the admin routes unreachable through the proxy.

Step 1: Bind the engine to loopback or the pod IP only

SGLang's --host defaults to 127.0.0.1 and --port to 30000, per the server arguments reference and the 0.5.23 source. The risk is the copy-pasted --host 0.0.0.0 in a Dockerfile or Helm chart. Find it first:

# Look for engines bound to all interfaces in your manifests and on hosts
grep -rnE -- '--host[ =]+(0\.0\.0\.0|::)' deploy/ charts/ docker/ 2>/dev/null
ss -ltnp | grep -E ':(30000|8000)\b'     # 30000 = SGLang default, 8000 = vLLM default

On a single host, bind to loopback and let the proxy in Step 3 be the only listener:

python -m sglang.launch_server \
  --model-path Qwen/Qwen3-8B \
  --host 127.0.0.1 --port 30000 \
  --api-key "$SGLANG_API_KEY" \
  --admin-api-key "$SGLANG_ADMIN_API_KEY"

In Kubernetes the engine has to listen on the pod IP, so the fence is a NetworkPolicy instead (Step 4). For vLLM, pass --host 127.0.0.1 explicitly rather than relying on a default, and for multi-node setups follow the vLLM security guide, which says to "always set VLLM_HOST_IP to a specific IP address rather than using defaults."

Step 2: Set separate API and admin keys in SGLang

SGLang has two keys. The 0.5.23 help text says --admin-api-key is for "sensitive management endpoints" and that admin endpoints then "do NOT accept --api-key." What it does not say plainly is what happens when you set only one. So I loaded sglang/srt/utils/auth.py from the 0.5.23 wheel and called its decide_request_auth() function for each combination. The route levels come from the decorators in http_server.py: 29 routes are normal (inference, /server_info, /v1/models), 43 are "admin optional" (/update_weights_from_disk, /load_lora_adapter_from_tensors, /dumper/{method}, /flush_cache, profiling and others).

# authmatrix.py: run SGLang 0.5.23's own auth decision against each key setup
import importlib.util, sys
spec = importlib.util.spec_from_file_location("sgl_auth", sys.argv[1])
m = importlib.util.module_from_spec(spec); sys.modules["sgl_auth"] = m; spec.loader.exec_module(m)
L = m.AuthLevel
for name, key, admin in [("no keys", None, None), ("--api-key only", "K", None),
                         ("--admin-api-key only", None, "A"), ("both keys", "K", "A")]:
    for label, path, lvl in [("/v1/chat/completions", "/v1/chat/completions", L.NORMAL),
                             ("/update_weights_from_disk", "/update_weights_from_disk", L.ADMIN_OPTIONAL)]:
        r = [m.decide_request_auth(method="POST", path=path, authorization_header=h,
                                   api_key=key, admin_api_key=admin, auth_level=lvl).allowed
             for h in (None, "Bearer K", "Bearer A")]
        print(f"{name:22s} {label:28s} none={r[0]} api={r[1]} admin={r[2]}")

Output on Python 3.12:

no keys                /v1/chat/completions         none=True api=True admin=True
no keys                /update_weights_from_disk    none=True api=True admin=True
--api-key only         /v1/chat/completions         none=False api=True admin=False
--api-key only         /update_weights_from_disk    none=False api=True admin=False
--admin-api-key only   /v1/chat/completions         none=True api=True admin=True
--admin-api-key only   /update_weights_from_disk    none=False api=False admin=True
both keys              /v1/chat/completions         none=False api=True admin=False
both keys              /update_weights_from_disk    none=False api=False admin=True

The same results as a table (allow means the request is let through, 401 means refused):

Key setup	Inference route, API key	Admin route, no key	Admin route, API key	Admin route, admin key
No keys	allow	allow	allow	allow
--api-key only	allow	401	allow	401
--admin-api-key only	allow (so does no key)	401	401	allow
Both keys	allow	401	401	allow

Three things follow. With no keys, every management endpoint is open to anyone who can reach the port. With only --api-key, every client that can run inference can also load weights from disk or push a LoRA adapter, which are exactly the routes CERT flagged. And with only --admin-api-key, inference and /server_info are unauthenticated, which is the setup behind CVE-2026-15977's key leak. Set both, and give clients only the API key.

Two more details from the source. /health, /ready and /metrics are always allowed regardless of keys, by design, for probes and Prometheus, so do not expose them publicly. And API keys are not supported in multi-tokenizer mode: the worker startup asserts that api_key is unset, so if you raise --tokenizer-worker-num above 1, authentication has to live entirely in the proxy.

For vLLM, --api-key (or VLLM_API_KEY) protects only the /v1, /v2, /inference and /cohere prefixes, according to the security guide. Routes such as /tokenize, /pause, /update_weights, /abort_requests and /metrics stay unauthenticated even with a key set. That is another reason the proxy allowlist is not optional.

Step 3: Put an allowlisting reverse proxy in front

The proxy terminates TLS, passes exact routes and refuses the rest. This config parsed with nginx -t on nginx 1.27.5:

# /etc/nginx/conf.d/inference.conf
upstream sglang {
    server 127.0.0.1:30000;   # engine bound to loopback only
    keepalive 32;
}

server {
    listen 8443 ssl;
    server_name inference.internal.example;

    ssl_certificate     /etc/nginx/tls/inference.crt;
    ssl_certificate_key /etc/nginx/tls/inference.key;

    client_max_body_size 8m;          # long prompts fit; multi-GB uploads do not

    # Exact-match allowlist
    location = /v1/chat/completions { proxy_pass http://sglang; include /etc/nginx/snippets/llm-proxy.conf; }
    location = /v1/completions      { proxy_pass http://sglang; include /etc/nginx/snippets/llm-proxy.conf; }
    location = /v1/embeddings       { proxy_pass http://sglang; include /etc/nginx/snippets/llm-proxy.conf; }
    location = /v1/models           { proxy_pass http://sglang; include /etc/nginx/snippets/llm-proxy.conf; }

    # Probes only from the node / load balancer range
    location = /health {
        allow 10.0.0.0/8;
        deny  all;
        proxy_pass http://sglang;
    }

    # Default deny
    location / { return 404; }
}

# /etc/nginx/snippets/llm-proxy.conf
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host $host;
proxy_buffering off;          # stream tokens as they arrive
proxy_read_timeout 600s;      # long generations

I ran it locally against a stub upstream on port 30000 and sent each request with curl --path-as-is:

Request	Status	Reached the engine?
POST /v1/chat/completions	200	yes
POST /v1/chat/completions?stream=true	200	yes
GET /v1/models	200	yes
POST /update_weights_from_disk	404	no
POST /load_lora_adapter_from_tensors	404	no
GET /server_info	404	no
POST /dumper/start, POST /flush_cache	404	no
POST /v1/chat/completions/../../update_weights_from_disk	404	no
GET /health from 127.0.0.1	403	no (outside the allow range)
POST /V1/chat/completions	200	yes, as /V1/...

The last row was a surprise until I checked the nginx location docs: on case-insensitive operating systems such as macOS, prefix matching ignores case, so the uppercase path got through and was forwarded unchanged. FastAPI routes are case-sensitive, so the engine would return 404 and nothing was exposed, but it is a reminder to test the proxy on the OS you deploy, not your laptop. Also keep the API key check in the engine; the proxy narrows what is reachable, it does not replace authentication.

Step 4: Fence the engine in Kubernetes

In a cluster, only the gateway should reach port 30000, and engine pods should only talk to each other for NCCL and rendezvous traffic. The vLLM guide is explicit that inter-node communication is "insecure by default" and belongs on an isolated network; the same applies to SGLang's --dist-init-addr and NCCL ports. This policy assumes default-deny is already in place, as in the NetworkPolicies guide. I have not applied it to a cluster for this guide, so run it through a staging namespace first.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: sglang-engine-ingress
  namespace: inference
spec:
  podSelector:
    matchLabels:
      app: sglang-engine
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: inference-gateway
      ports:
        - protocol: TCP
          port: 30000
    - from:
        - podSelector:
            matchLabels:
              app: sglang-engine      # peer engine pods (multi-node tensor parallel)
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: sglang-engine
    - to:                              # DNS
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: kube-system
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

The egress rules deliberately block the internet, including Hugging Face. Pre-cache weights on a volume, as in the vLLM on Kubernetes guide. An engine that cannot call out cannot exfiltrate either, and that is the property you want if an RCE does land.

Step 5: Reduce what the engine can do and what it holds

• Restrict remote media. SGLang 0.5.23 has --allowed-media-domains; its help text says that "when unset, remote media from any domain is allowed." Set it, or block image_url at the gateway, to close the SSRF class in CVE-2026-15974. vLLM has an equivalent --allowed-media-domains flag.
• Leave --trust-remote-code off (its default) unless you have read the model's custom code.
• Consider SGLANG_USE_PICKLE_IPC=false. CERT lists it as a mitigation, and 0.5.23 still defaults it to true. I have not tested the engine with it off, so benchmark before you roll it out.
• Fix CORS on vLLM. GHSA-vvpg-rrww-wpmh says the default allowed origins were *; upgrade to 0.32.0 or later and set origins explicitly.
• Strip secrets from the engine's environment. No cloud credentials, no long-lived Hugging Face write tokens. NEAR found customer API keys in logs; do not log request bodies (--log-requests is off by default in SGLang) on servers that see real traffic.
• Run engines in their own node pool and service account, with no cloud metadata access.

Troubleshooting

Symptom	Likely cause	Fix
Clients get 401 after adding keys	Client sends no Authorization: Bearer header	Set the API key in the OpenAI SDK's api_key field
Admin call with the API key returns 401	Both keys set, admin routes require the admin key	Use the admin key from the host or bastion only
SGLang exits at startup with "API key is not supported in multi-tokenizer mode"	--api-key with --tokenizer-worker-num above 1	Drop to one tokenizer worker or move auth to the proxy
Streaming responses arrive in one block	Proxy buffering	proxy_buffering off; in the location
Long generations cut off at 60 seconds	Default nginx read timeout	Raise proxy_read_timeout
Kubernetes probes fail after the NetworkPolicy	Kubelet traffic blocked by your CNI's policy handling	Allow the node CIDR to port 30000 for /health
Multi-node engine hangs at init	NCCL or rendezvous ports blocked between pods	Keep the peer-pod ingress and egress rules
Engine cannot download the model	Egress blocked on purpose	Pre-cache weights on a volume

FAQ

Does SGLang's --api-key protect admin endpoints?

Yes, but too broadly. In SGLang 0.5.23, if only --api-key is set, that same key unlocks the admin routes such as /update_weights_from_disk and /load_lora_adapter_from_tensors. Set --admin-api-key as well, and the admin routes then accept only the admin key.

Is it safe to expose vLLM or SGLang directly to the internet?

No. Both projects' security guidance and CERT/CC recommend restricting network access. vLLM's API key leaves several routes unauthenticated, SGLang leaves health and metrics open by design, and both have had code execution advisories in 2026. Put an authenticating, allowlisting proxy in front.

Is there a patch for the SGLang flaw used against NEAR AI?

Not as of 13 November 2026. NEAR AI has not named a CVE or version, and the SGLang repository has no published advisories. The controls in this guide reduce exposure while you wait.

When I review a self-hosted inference setup, I check three things first: what is bound to 0.0.0.0, whether one key can load weights, and whether the engine pod can reach the internet. Most findings come from those. Docs worth keeping open: SGLang server arguments, vLLM security guide and CERT VU#281278.

Related: NEAR AI Breached Through an SGLang RCE, How to Deploy vLLM on Kubernetes with GPUs and AI Agent Security Incident Log.