How to Fix a Kubernetes Service Connection Refused or Timeout (2026)

Nov 10, 2026 · 11 min · Ajay Kumar

This guide is for anyone staring at curl: (7) Failed to connect ... Connection refused or a request that hangs until it times out when calling a Kubernetes Service. By the end you will be able to tell which layer is failing in a few minutes and fix the usual causes: a selector that matches no Pods, an empty EndpointSlice, a wrong targetPort, an app bound to localhost, a failing readiness probe, a NetworkPolicy, or kube-proxy. Commands were checked against Kubernetes v1.37.2, kubectl 1.37 and the netshoot v0.16 image. You need kubectl access to the namespace, and for the kube-proxy step, permission to run a debug Pod on a node. If you want the background first, the companion post Kubernetes Networking Explained (2026) covers how Services, EndpointSlices and kube-proxy fit together.

The examples use a Service called web in namespace shop. Substitute your own. Output shown in comments under the commands is illustrative of the expected shape, not captured from a live cluster.

export NS=shop SVC=web

Step 1: Reproduce from inside the cluster and read the error

Do not debug from your laptop through a port-forward or an Ingress. Start a throwaway Pod in the same namespace with real tools in it, and call the Service by its full DNS name.

# netshoot has curl, dig, nc, tcpdump, iptables, nft and conntrack.
kubectl -n "$NS" run tmp-shell --rm -it --restart=Never \
  --image=nicolaka/netshoot:v0.16 -- bash

# Inside the shell:
dig +short web.shop.svc.cluster.local      # should print the ClusterIP
curl -sv --max-time 5 http://web.shop.svc.cluster.local:80/

Now read the failure carefully, because the two errors point at different layers:

  • Connection refused arrives instantly. Something actively sent back a TCP RST or ICMP reject. With kube-proxy in iptables mode, a Service with zero endpoints gets a REJECT rule commented "has no endpoints" (kube-proxy source), and a Pod that has nothing listening on the target port also answers with a reset. So refused usually means endpoints or ports.
  • Timeout means packets vanished. That is a NetworkPolicy drop, a CNI routing problem between nodes, a stale kube-proxy, or traffic sent to a node with no local endpoint.
  • Could not resolve host is DNS, not the Service. Jump to Step 7.
Service unreachable: refused or timeout? curl -v from a netshoot Pod to the Service Connection refused (instant RST) Timeout (packets dropped) EndpointSlice empty? selector mismatch or Pods not ready NetworkPolicy dropping ingress? or CNI not routing between nodes Endpoints present? wrong targetPort or app bound to 127.0.0.1 kube-proxy not syncing rules? or externalTrafficPolicy: Local on NodePort Test the Pod IP directly before the Service IP: if the Pod IP fails too, the Service is not the problem.
The triage order used in this guide. Refused points at endpoints and ports; a timeout points at policy, routing or kube-proxy. Based on the Kubernetes Debug Services task page and kube-proxy behaviour in v1.37.

Step 2: Check that the selector matches running Pods

The most common cause by far. A Service finds Pods only by label, and a single typo (app: web vs app: web-api, or a label that changed in a Helm upgrade) leaves it with nothing behind it.

# What the Service selects
kubectl -n "$NS" get svc "$SVC" -o jsonpath='{.spec.selector}{"\n"}'
# {"app":"web"}

# Do any Pods carry exactly those labels?
kubectl -n "$NS" get pods -l app=web -o wide --show-labels
# No resources found in shop namespace.   <- that is your bug

If the second command returns nothing, fix whichever side is wrong. Changing the Service selector is safe; changing Deployment spec.selector is not (it is immutable, so you would have to recreate the Deployment). Also confirm you are in the right namespace: a Service only selects Pods in its own namespace.

Step 3: Look at the EndpointSlices, not just the Service

The Service object never lists Pod IPs. The EndpointSlice controller does that, in objects labelled with the Service name. The v1 Endpoints API was deprecated in v1.33, so read slices.

kubectl -n "$NS" get endpointslices -l kubernetes.io/service-name="$SVC" -o wide
# NAME        ADDRESSTYPE   PORTS   ENDPOINTS                AGE
# web-7xk2p   IPv4          8080    10.244.1.7,10.244.2.3    3d

# See readiness per endpoint
kubectl -n "$NS" get endpointslices -l kubernetes.io/service-name="$SVC" \
  -o jsonpath='{range .items[*].endpoints[*]}{.addresses[0]}{"  ready="}{.conditions.ready}{"\n"}{end}'
# 10.244.1.7  ready=true
# 10.244.2.3  ready=false

Three outcomes:

  • No slice, or <unset> endpoints: go back to Step 2, or the Service has no selector at all (a selectorless Service needs EndpointSlices you create yourself).
  • Endpoints present but ready=false: the Pods exist and match, but are failing readiness. kube-proxy will not send traffic to them. Go to Step 5.
  • Endpoints present and ready: the routing data is right, so the problem is the port or the process. Go to Step 4.

The PORTS column is worth a glance here too: it shows the resolved target port. If it says 80 and your app listens on 8080, you have already found it.

Step 4: Fix targetPort vs containerPort, and localhost binding

Three port fields exist and only one of them decides where packets go. From the Service docs: port is what clients call on the Service, targetPort is where traffic is sent on the Pod (number or port name), and containerPort in the Pod spec is informational. If targetPort is omitted, it defaults to the same value as port.

# Service: clients call web:80, traffic lands on the container's port named "http"
apiVersion: v1
kind: Service
metadata:
  name: web
  namespace: shop
spec:
  selector:
    app: web
  ports:
    - name: http
      port: 80
      targetPort: http      # must match a containerPort name in the Pod, or use 8080
      protocol: TCP
---
# Matching container port in the Deployment's Pod template
#   ports:
#     - name: http
#       containerPort: 8080

Prove what the process is actually listening on, from inside the Pod's network namespace, with an ephemeral debug container. This works even on distroless images that have no shell:

POD=$(kubectl -n "$NS" get pods -l app=web -o jsonpath='{.items[0].metadata.name}')
kubectl -n "$NS" debug -it "pod/$POD" --image=nicolaka/netshoot:v0.16 \
  --target=web --profile=netadmin -- ss -lntp
# State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port
# LISTEN 0      4096        127.0.0.1:8080       0.0.0.0:*     <- bound to localhost only

--target shares the process namespace with the named container, and --profile accepts general (the default), baseline, restricted, netadmin or sysadmin in current kubectl. If the listener shows 127.0.0.1:8080, the app is reachable from its own Pod and refused from everywhere else. Configure it to bind 0.0.0.0 (often a HOST, --bind or server.address setting). This catches dev-server defaults all the time.

Then test the Pod IP directly from your netshoot shell from Step 1:

curl -sv --max-time 5 http://10.244.1.7:8080/

If the Pod IP works and the Service IP does not, the problem is between the Service and the Pod (ports, kube-proxy). If the Pod IP fails too, stop looking at the Service.

Step 5: Fix readiness so Pods become endpoints

A Pod that fails its readiness probe stays out of rotation. When every replica is unready, the Service effectively has no endpoints and you get refused.

kubectl -n "$NS" get pods -l app=web
# NAME                   READY   STATUS    RESTARTS   AGE
# web-6d9f8c7b5d-abcde   0/1     Running   0          4m

kubectl -n "$NS" describe pod web-6d9f8c7b5d-abcde | grep -A3 -i readiness
# Readiness probe failed: Get "http://10.244.1.7:8081/ready": connection refused

The usual culprits are a probe pointing at the wrong port or path, or a dependency check inside /ready that fails when a database is slow. The probe failure guide goes through each case. Do not "fix" this by setting publishNotReadyAddresses: true on a normal Service; that is meant for headless Services used by StatefulSet peer discovery.

Step 6: Rule out NetworkPolicy

If you get a timeout and the endpoints look healthy, check for policies. Once any NetworkPolicy selects a Pod for ingress, everything not explicitly allowed to that Pod is dropped, silently.

kubectl -n "$NS" get networkpolicy
kubectl -n "$NS" describe networkpolicy

Look for a default-deny policy and confirm an allow rule matches your client's namespace and Pod labels and the target port (policies match the Pod port, 8080 here, not the Service port 80). A quick test: run the netshoot Pod with the same labels as a known-good client. Remember the reverse case too: if your CNI does not enforce policy, the Flannel daemon for example does not natively, policies do nothing and are not your problem. My NetworkPolicy guide has working allow-rule examples.

Step 7: Check DNS resolution

If curl says it could not resolve the host, or dig returned nothing, the Service may be fine. The Debug Services task page suggests trying the short, namespaced and fully qualified names:

nslookup web                                  # works only from the same namespace
nslookup web.shop
nslookup web.shop.svc.cluster.local
cat /etc/resolv.conf                          # nameserver should be the cluster DNS Service IP

If only the short name fails, your client is in a different namespace. If all fail, follow the CoreDNS troubleshooting guide.

Step 8: Check kube-proxy and the node rules

If endpoints are ready, the Pod IP works, and the ClusterIP still fails, look at the thing that turns the ClusterIP into a Pod IP. On kubeadm-style clusters kube-proxy is a DaemonSet:

kubectl -n kube-system get pods -l k8s-app=kube-proxy -o wide
kubectl -n kube-system logs -l k8s-app=kube-proxy --tail=50 | grep -i -E "error|fail"
kubectl -n kube-system get configmap kube-proxy -o jsonpath='{.data.config\.conf}' | grep '^mode'

To see the rules on a specific node, use a node debug Pod, which runs in the host network namespace:

kubectl debug node/worker-1 -it --image=nicolaka/netshoot:v0.16 --profile=sysadmin

# iptables mode: one KUBE-SERVICES rule per Service port, a KUBE-SVC chain, KUBE-SEP per endpoint
iptables-save | grep 'shop/web'

# nftables mode: rules live in the "kube-proxy" table
nft list table ip kube-proxy | grep 'shop/web'   # chain names embed namespace/name

No rules for your Service means kube-proxy is not syncing on that node; restart its Pod and read the logs. If your cluster uses Cilium in kube-proxy-free mode, these commands show nothing useful. Use kubectl -n kube-system exec ds/cilium -- cilium-dbg service list instead (command reference). IPVS clusters should note that IPVS mode is deprecated and scheduled to be disabled by default in v1.40.

One more node-level timeout: a NodePort or LoadBalancer Service with externalTrafficPolicy: Local only answers on nodes that run a ready endpoint. Traffic sent to any other node is dropped by design. Check which nodes host the Pods (kubectl get pods -o wide) and make sure your load balancer health checks use the Service's healthCheckNodePort.

Troubleshooting

Symptom Likely cause Fix
Instant connection refused, EndpointSlice empty Selector does not match Pod labels, or wrong namespace Compare spec.selector with kubectl get pods --show-labels; fix the Service selector
Refused, endpoints listed but ready=false Readiness probe failing Fix probe port, path or dependency check; see the probe guide
Refused, endpoints ready targetPort does not match the listening port Set targetPort to the real port or its container port name
Refused from other Pods, works via kubectl exec localhost App bound to 127.0.0.1 Bind to 0.0.0.0
Timeout from some namespaces only NetworkPolicy default deny Add an ingress allow rule for the client labels and Pod port
Pod IP times out from another node CNI routing, MTU or overlay broken Check CNI Pods and logs; test Pod-to-Pod across nodes
Pod IP works, ClusterIP times out kube-proxy not running or not syncing Check kube-proxy logs; inspect iptables-save or nft list table ip kube-proxy
NodePort times out on some nodes externalTrafficPolicy: Local with no endpoint on that node Target nodes with endpoints, or use Cluster
NodePort works on one IP but not another after nftables switch nftables mode listens on primary node IPs only Set nodePortAddresses explicitly
Could not resolve host DNS or wrong namespace in short name Use name.namespace; debug CoreDNS

FAQ

Why does my Kubernetes Service say connection refused?

Refused means something answered and rejected the connection. In practice that is a Service with no ready endpoints (kube-proxy rejects traffic for it), a targetPort that does not match the port the app listens on, or an app listening only on localhost. Check the EndpointSlice first, then the port.

What is the difference between port, targetPort and containerPort?

port is the port clients use on the Service IP. targetPort is the port on the Pod that traffic is forwarded to, and it defaults to port if unset. containerPort documents what the container exposes and lets you name it, but it does not open or redirect anything.

How do I debug a Service on a distroless image with no shell?

Use an ephemeral container: kubectl debug -it pod/NAME --image=nicolaka/netshoot:v0.16 --target=CONTAINER. It joins the Pod's network namespace, so ss, curl and tcpdump see exactly what the app sees, without rebuilding the image.

Should I still use kubectl get endpoints?

The v1 Endpoints API was deprecated in Kubernetes v1.33 in favour of EndpointSlices. It still works and will not be removed, but tools and scripts should use kubectl get endpointslices -l kubernetes.io/service-name=NAME.

What I check first in production is boring and fast: kubectl get endpointslices for the Service, then curl to one Pod IP from a netshoot Pod. Those two commands split the problem in half within a minute, and the answer is usually a label or a port, not the network. Only after that do I go near node rules. For the official references, keep the Kubernetes Debug Services page, the Service concepts page and the virtual IPs and service proxies reference open in tabs.

Related: How to Troubleshoot Kubernetes DNS and CoreDNS: Step by Step (2026), How to Write Kubernetes NetworkPolicies: Default Deny to Egress (2026) and How to Fix Kubernetes Readiness and Liveness Probe Failed Errors (2026).

Spot an error or something out of date? Email me and I will fix it and log it on the corrections page.

I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.

Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.


Related