Skip to content
pilots
Dashboard
All pages

Services and releases

A service is one or more machines behind a URL that never changes. Everything that happens to it afterwards, a deploy, a rollback, a scale, is a change of which machines sit behind that address.

What is running

pilot service ls
pilot service info web
pilot service releases web

service info gives the health check, the environment KEYS, the domain and the replica ids. It never returns an environment VALUE, which is deliberate and is not a gap to be worked around (secrets says why).

A release, and the gate in front of it

A deploy builds a root filesystem, creates a release, starts a replica from it, and lets that release take traffic only once the replica has answered its health check. Until then the previous release is still serving, and if the new replica never answers, the previous release simply stays.

That is the whole reason a deploy takes as long as your application takes to answer for the first time. A health_gate_failed answer carries the service, the replica, the release, the grace it was given and the last thing the probe saw. When the probe saw a refused connection it means one of two things and almost nothing else: the app is listening on the wrong port, or it bound the loopback address instead of all interfaces.

Rolling back

pilot service rollback web

Rolling back puts the previous release's machines back in front of the URL. It changes what is live for everyone, so confirm it with whoever owns the service first, and an agent must ask before calling it. pilot service releases lists what there is to go back to, newest first, with whether each was healthy and which build it came from.

Replicas and scaling

pilot service scale web --replicas 3
pilot service scale web --vcpus 2 --mem 2048

A SIZE change replaces the replicas one at a time at the same release: a replica comes up at the new size, passes the same gate a deploy passes, and only then is an old one retired. Nothing is rebuilt and no request is dropped. A volume-backed service is the exception, because the replacement cannot mount the volume until the old machine has let go, and requests arriving in that window are held rather than refused.

  • A replica with no traffic suspends after about half a minute of quiet, and the next request wakes it. That is the default, and while it sleeps only its disk is charged.
  • min_machines_running in the compose file keeps replicas resident. A queue consumer or a scheduler with nothing connected to it needs one, because nothing is going to wake it.
  • soft_limit, also a compose key, is how many concurrent requests a replica takes before more start, 20 by default. When every replica is at it, the autoscaler adds as many as the load in flight needs in one pass, up to max_machines_running, each on the host holding the fewest of the service's replicas.
  • A change to these keys alone keeps the running release and applies in place, with no restart. pilot service set web --min-machines 1 --max-machines 6 does the same by hand.
  • A volume-backed service runs exactly ONE replica and asking for more is refused (volumes).

Services with no URL

A service that has nothing to serve, a worker or a database, asks for no address with private: true. Its peers still reach it at <name>.internal, and nothing outside the app can.

Reading a replica

pilot logs web
pilot logs web --tail 50

pilot logs prefixes every line with the replica it came from, so one command covers a service however many machines it has. It is the console, so it needs no shell in the image and no logging agent inside it. Do not exec a replica to tail a file.

Finding services by label

pilot service ls --label owner=crisp --label project=p_1
pilot api '/v1/services?label=owner=crisp' --json

A label set at create is how a product built on pilots finds its own services again without keeping a second index: every label given must match, and the fleet does the filtering. The SDKs take the same filter as labels on services.list and machines.list, and the MCP tools as labels on list_services and list_machines.

Deleting a service

pilot service delete web
pilot service delete web --delete-volumes -y
pilot api -X DELETE /v1/services/svc_8f3a --json

A delete removes everything the service owns: every replica of every release, the parked rollback target included, wherever each one runs. It releases the custom domains, which are then free to add elsewhere, and drops the releases and the deploy history. Its address stops answering. The answer lists what went: machines, domains, volumes_deleted and volumes_kept.

  • Volumes are the one thing a delete cannot give back, so they are detached and KEPT unless you pass --delete-volumes (?delete_volumes=true on the API, deleteVolumes in the SDK). A kept volume is listed by pilot volume ls and deleted later with pilot volume destroy.
  • It is idempotent. A service that is already gone answers deleted: false, the same answer an id you cannot see gets, and a delete that stopped part way because a host did not answer (unavailable, 503) is finished by sending it again.
  • A delete during a deploy is refused with conflict: let the rollout finish, then delete.

Answers worth knowing

Answer What it means Do
The release id moved The new release is serving. Nothing.
health_gate_failed The replica never answered. Diagnose the replica named in the answer. Deploying again changes nothing.
conflict A rollout is already running on this service. Wait for it. The service shows it.
not_found Wrong id, or this key belongs to another organization. pilot service ls shows what the key can see.