All pages
Services and releases
A service is one or more machines behind a URL that never changes. Everything that happens to it afterwards, a deploy, a rollback, a scale, is a change of which machines sit behind that address.
What is running
pilot service ls
pilot service info web
pilot service releases web
service info gives the health check, the environment KEYS, the
domain and the replica ids. It never returns an environment VALUE, which is deliberate and
is not a gap to be worked around (secrets says why).
A release, and the gate in front of it
A deploy builds a root filesystem, creates a release, starts a replica from it, and lets that release take traffic only once the replica has answered its health check. Until then the previous release is still serving, and if the new replica never answers, the previous release simply stays.
That is the whole reason a deploy takes as long as your application takes to
answer for the first time. A health_gate_failed answer carries the service, the
replica, the release, the grace it was given and the last thing the probe saw. When the
probe saw a refused connection it means one of two things and almost nothing else: the app
is listening on the wrong port, or it bound the loopback address instead of all
interfaces.
Rolling back
pilot service rollback web
Rolling back puts the previous release's machines back in front of the URL.
It changes what is live for everyone, so confirm it with whoever owns the service first,
and an agent must ask before calling it. pilot service releases lists what
there is to go back to, newest first, with whether each was healthy and which build it
came from.
Replicas and scaling
pilot service scale web --replicas 3
pilot service scale web --vcpus 2 --mem 2048
A SIZE change replaces the replicas one at a time at the same release: a replica comes up at the new size, passes the same gate a deploy passes, and only then is an old one retired. Nothing is rebuilt and no request is dropped. A volume-backed service is the exception, because the replacement cannot mount the volume until the old machine has let go, and requests arriving in that window are held rather than refused.
- A replica with no traffic suspends after about half a minute of quiet, and the next request wakes it. That is the default, and while it sleeps only its disk is charged.
min_machines_runningin the compose file keeps replicas resident. A queue consumer or a scheduler with nothing connected to it needs one, because nothing is going to wake it.soft_limit, also a compose key, is how many concurrent requests a replica takes before more start, 20 by default. When every replica is at it, the autoscaler adds as many as the load in flight needs in one pass, up tomax_machines_running, each on the host holding the fewest of the service's replicas.- A change to these keys alone keeps the running release and applies in place, with
no restart.
pilot service set web --min-machines 1 --max-machines 6does the same by hand. - A volume-backed service runs exactly ONE replica and asking for more is refused (volumes).
Services with no URL
A service that has nothing to serve, a worker or a database, asks for no
address with private: true. Its peers still reach it at
<name>.internal, and nothing outside the app can.
Reading a replica
pilot logs web
pilot logs web --tail 50
pilot logs prefixes every line with the replica it came from, so
one command covers a service however many machines it has. It is the console, so it needs
no shell in the image and no logging agent inside it. Do not exec a replica to tail a
file.
Finding services by label
pilot service ls --label owner=crisp --label project=p_1
pilot api '/v1/services?label=owner=crisp' --json
A label set at create is how a product built on pilots finds its own
services again without keeping a second index: every label given must match, and the
fleet does the filtering. The SDKs take the same filter as labels on
services.list and machines.list, and the MCP tools as
labels on list_services and list_machines.
Deleting a service
pilot service delete web
pilot service delete web --delete-volumes -y
pilot api -X DELETE /v1/services/svc_8f3a --json
A delete removes everything the service owns: every replica of every
release, the parked rollback target included, wherever each one runs. It releases the
custom domains, which are then free to add elsewhere, and drops the releases and the
deploy history. Its address stops answering. The answer lists what went: machines, domains,
volumes_deleted and volumes_kept.
- Volumes are the one thing a delete cannot give back, so they are detached and
KEPT unless you pass
--delete-volumes(?delete_volumes=trueon the API,deleteVolumesin the SDK). A kept volume is listed bypilot volume lsand deleted later withpilot volume destroy. - It is idempotent. A service that is already gone answers
deleted: false, the same answer an id you cannot see gets, and a delete that stopped part way because a host did not answer (unavailable, 503) is finished by sending it again. - A delete during a deploy is refused with
conflict: let the rollout finish, then delete.
Answers worth knowing
| Answer | What it means | Do |
|---|---|---|
| The release id moved | The new release is serving. | Nothing. |
health_gate_failed
| The replica never answered. | Diagnose the replica named in the answer. Deploying again changes nothing. |
conflict
| A rollout is already running on this service. | Wait for it. The service shows it. |
not_found
| Wrong id, or this key belongs to another organization. |
pilot service ls shows what the key can see.
|