Companion to the 13-part UmimaClean stack series and the 5-part VPS setup series. Source: github.com/r3p-dev/skripsi.
The stack series described how UmimaClean is built. This one is about getting it onto a server and keeping it there, with git push origin master as the only step I perform by hand.
It is a different kind of problem. Nothing here is clever. What it is instead is a long list of small things that each break the deploy in a way that looks like an application bug, and the whole value of writing it down is that the list exists somewhere other than my memory.
The shape of the finished thing: a push to master runs the test suite, builds a container image on GitHub’s runner, pushes it to GHCR, then connects to the VPS over SSH to migrate and restart. The server never builds anything and never holds a git checkout.
What This Assumes
A VPS set up the way the VPS series describes: Debian, a non-root admin user with lingering enabled, rootless Podman, an apps Podman network, and Caddy already running on it as a Quadlet unit terminating HTTPS. If your server looks different, the Quadlet units below are the part you will have to translate.
Everything runs as the unprivileged admin user. Nothing in this post needs sudo except the reboot test.
What the Application Actually Needs
Before writing a single unit file, it is worth reading the repository for its deployment requirements rather than guessing them. For UmimaClean the list comes out of four files.
package.json sets "engines": { "node": ">=24.0.0" } and pins packageManager to pnpm 11.6.0. Both matter — the second one because Corepack will refuse to run a different pnpm than the one the lockfile was written with.
start/env.ts validates every environment variable at boot. Not “reads” — validates, with a schema, before the HTTP server starts. A missing FONNTE_API_KEY is a crash, not a warning, even on a page that never sends a WhatsApp message. This is the single most common reason a first deploy dies three seconds after starting.
config/database.ts hardcodes connection: 'pg'. Postgres is not a preference here, it is the only configured client.
config/limiter.ts defaults to LIMITER_STORE=database, which means the rate limiter needs the rate_limits table to exist before the first request arrives — not before the first rate-limited request.
Two more that are easy to miss:
config/drive.ts points the fs disk at app.makePath('storage') with visibility: 'private' and serveFiles: true. That directory lives inside the application root, which in a container means inside the image. Anything written there disappears on the next deploy unless it is mounted in from outside.
config/transmit.ts sets transport: null. Server-sent events are held in the memory of one process, so this application runs as exactly one container. Not two behind a load balancer — one. Part 6 explains why that was an acceptable trade for a single shop.
What node ace build Produces
AdonisJS builds to a standalone JavaScript application. Worth being precise about what that means, because the Containerfile depends on it:
node ace buildThis does three things. It compiles TypeScript to build/. It runs the Vite build hook declared in adonisrc.ts, which compiles the React frontend into public/assets. And it copies everything matched by metaFiles — the Edge templates and all of public/ — into the build output.
What it deliberately does not do is install dependencies. The build/ directory contains a package.json and the lockfile, and it is your job to run a production install inside it:
cd build
pnpm install --prod
node bin/server.jsThat two-step shape is what makes a clean two-stage image possible: the first stage owns the entire toolchain, the second owns a build/ directory and production dependencies only. Whether the second stage gets those dependencies by installing them fresh or by inheriting a pruned tree from the first is a detail — the Containerfile below takes the second route, for reasons that become clearer once you have seen both.
One detail specific to this repository. pnpm 11 reads pnpm-workspace.yaml, and here that file carries both overrides and allowBuilds:
allowBuilds:
'@swc/core': true
esbuild: true
overrides:
'eslint-plugin-react>eslint': '^10.4.0'The assembler copies package.json and the lockfile into build/, but knows nothing about pnpm-workspace.yaml. There are two ways to deal with that, and the Containerfile below takes the one that never has to think about it.
The Containerfile
This lives at the repository root and is committed, because the thing that builds it is a GitHub runner rather than the server. A Containerfile sitting only on the VPS was fine when I deployed by hand; the moment CI does the building, the build definition has to travel with the code it builds.
# =========================
# Base
# =========================
FROM docker.io/library/node:24-slim AS base
ENV PNPM_HOME="/pnpm"
ENV PATH="$PNPM_HOME/bin:$PATH"
RUN corepack enable
WORKDIR /app
# =========================
# Dependencies
# =========================
COPY package.json pnpm-lock.yaml pnpm-workspace.yaml ./
RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm fetch
RUN --mount=type=cache,id=pnpm,target=/pnpm/store \
pnpm install --offline --frozen-lockfile
# =========================
# Build
# =========================
COPY . .
RUN pnpm exec node ace build
RUN pnpm prune --prod
# =========================
# Production
# =========================
FROM docker.io/library/node:24-slim AS prod
WORKDIR /app
ENV NODE_ENV=production
ENV TZ=UTC
COPY --from=base /app/build ./
COPY --from=base /app/package.json ./
COPY --from=base /app/node_modules ./node_modules
EXPOSE 3333
CMD ["node", "bin/server.js"]A few lines deserve explanation.
node:24-slim rather than Alpine. Debian with glibc, matching the host, and it already carries the timezone database — TZ=Asia/Jakarta resolves correctly out of the box. The Alpine equivalent needs an explicit apk add tzdata or TZ is silently ignored, which is a delightful way to spend an evening after reading the export timezone post. TZ=UTC matches .env.example; every date is stored and computed in UTC and formatted into Asia/Jakarta at the edges.
corepack enable installs the exact pnpm named in package.json. Without it you get whatever pnpm the base image happens to ship, and --frozen-lockfile starts failing for reasons that have nothing to do with your dependencies.
pnpm fetch before COPY . . is a stronger version of the ordering trick from Part 5 of the VPS series. fetch populates the store from the lockfile alone, so the expensive layer depends on pnpm-lock.yaml and nothing else — not even package.json contents beyond what the lockfile pins. --offline then guarantees the install cannot quietly reach the network and resolve something newer.
pnpm prune --prod in the build stage is what removes the cd build && pnpm install step that the AdonisJS docs describe. The dependencies are already installed and already correct; pruning them in place and copying the tree forward is faster than resolving a second time, and it sidesteps the pnpm-workspace.yaml problem entirely — the production stage never runs an install, so it never needs the file that governs installs.
COPY --from=base /app/build ./ flattens the build directory into /app, so the application root inside the container is /app and app.makePath('storage') resolves to /app/storage. Remember that path — the mount below has to match it exactly.
package.json is not optional at runtime. It is easy to read that line as belt-and-braces next to a whole node_modules, but this project’s package.json carries the imports map — #controllers/*, #models/*, #services/*, twenty-odd of them — and Node resolves those subpath imports at require time, from the nearest package.json. Without it every #-prefixed import in the compiled output fails with ERR_PACKAGE_IMPORT_NOT_DEFINED the moment a route is hit.
On --ignore-ts-errors. The assembler accepts it, and it is tempting, because a type error stopping a deploy at 11pm is infuriating. I would keep it out of this file. node ace build type-checking the codebase is the last gate before an image exists, and the CI job below already runs pnpm typecheck separately — if that job is green, the flag changes nothing, and if it is red, the flag is the only reason a broken build shipped. Use it when you deliberately want to deploy past a known type error, from a branch, on purpose.
Next to it, a .dockerignore:
node_modules
build
tmp
storage
public/assets
.git
.env
.env.*
testsstorage is on that list for a reason. If you have ever run the app locally, that directory holds customer photos. Sending them into a build context is slow and copies data onto the server that has no business being there.
Environment
None of the secrets belong in the image. They go in a file that only admin can read:
mkdir -p ~/apps/umimaclean
nano ~/apps/umimaclean/.env
chmod 600 ~/apps/umimaclean/.envTZ=UTC
PORT=3333
HOST=0.0.0.0
NODE_ENV=production
LOG_LEVEL=info
APP_KEY=<generated, see below>
APP_URL=https://umimaclean.example.com
SESSION_DRIVER=cookie
DRIVE_DISK=fs
LIMITER_STORE=database
DB_HOST=umimaclean-db
DB_PORT=5432
DB_USER=umimaclean
DB_PASSWORD=<long random string>
DB_DATABASE=umimaclean
MIDTRANS_MERCHANT_ID=
MIDTRANS_SERVER_KEY=
FONNTE_API_KEY=Four of those lines differ from .env.example in ways that matter.
HOST=0.0.0.0. The example says localhost, which is correct for development and fatal in a container — the app would bind to the loopback interface of its own network namespace and be unreachable from Caddy. As covered in the VPS series, this does not expose anything, because the container publishes no host port at all.
APP_URL must be the public HTTPS URL. It is what signed URLs are built from — including the password-reset links sent over WhatsApp in Part 9. Get it wrong and every reset link points somewhere useless.
DB_HOST is a container name, resolved by Podman’s DNS on the shared network. There is no host, no port, and no IP address to keep in sync.
The Midtrans and Fonnte keys cannot be left empty, even if you are only bringing the app up to look at it. Env.schema.secret() requires a non-empty value. This is where the boot loop comes from.
Generate the app key anywhere you have the project checked out:
node ace generate:keyCopy the value into .env and then leave it alone forever. SESSION_DRIVER=cookie means session data lives in encrypted cookies signed with that key, and every signed URL derives from it too. Rotating it logs out every user and invalidates every reset link still in someone’s WhatsApp. That is occasionally what you want; it is never something you want by accident.
Postgres as a Quadlet Unit
A volume first, so the data outlives the container:
nano ~/.config/containers/systemd/umimaclean-db.volume[Volume]
VolumeName=umimaclean-dbThen the database itself:
nano ~/.config/containers/systemd/umimaclean-db.container[Unit]
Description=UmimaClean PostgreSQL
Wants=network-online.target
After=network-online.target
[Container]
ContainerName=umimaclean-db
Image=docker.io/library/postgres:18-alpine
AutoUpdate=registry
Network=apps.network
Volume=umimaclean-db.volume:/var/lib/postgresql/data
Environment=POSTGRES_USER=umimaclean
Environment=POSTGRES_DB=umimaclean
Environment=POSTGRES_PASSWORD=<same as DB_PASSWORD>
Environment=PGDATA=/var/lib/postgresql/data/pgdata
HealthCmd=pg_isready -U umimaclean -d umimaclean
HealthInterval=10s
HealthRetries=5
HealthStartPeriod=30s
Notify=healthy
[Service]
Restart=always
[Install]
WantedBy=default.targetNo PublishPort here either. The database is reachable from the apps network and nowhere else — not from the host, not from the internet, not from a port that seemed harmless at the time.
Notify=healthy is the line that saves you from a race. Without it, systemd considers the unit started as soon as the container is running, and the application connects to a Postgres that has not finished initialising on first boot. With it, the unit is not “started” until pg_isready succeeds, so an After= dependency actually means what it looks like it means.
When you need a psql prompt:
podman exec -it umimaclean-db psql -U umimaclean -d umimacleanThe Application Unit
The uploads directory is a plain directory on the host rather than a named volume, so that backups and the occasional “what is actually in there” are ordinary file operations:
mkdir -p ~/apps/umimaclean/storagenano ~/.config/containers/systemd/umimaclean.container[Unit]
Description=UmimaClean application
Wants=network-online.target
After=network-online.target umimaclean-db.service
Requires=umimaclean-db.service
[Container]
ContainerName=umimaclean
Image=ghcr.io/r3p-dev/skripsi:master
Network=apps.network
Volume=%h/apps/umimaclean/storage:/app/storage:Z
EnvironmentFile=%h/apps/umimaclean/.env
[Service]
Restart=always
[Install]
WantedBy=default.targetThe image reference is the one line that changed when I moved to CI. It used to be localhost/umimaclean:latest, built on the server. Now it names a tag in GitHub’s registry that the pipeline pushes to, and the deploy step pulls it explicitly before restarting — so systemd itself never has to reach the network to start the service.
There is no AutoUpdate= line, deliberately. AutoUpdate=registry would let the machine pull and restart on its own schedule, which sounds convenient and means production changing without a deploy having happened. The pipeline is the only thing that moves this container.
%h expands to the user’s home directory, which keeps the unit copy-pasteable between machines and users. The :Z suffix relabels the bind mount for SELinux; on Debian it is a harmless no-op, and on Fedora or RHEL its absence is why the container gets permission denied on a directory that looks perfectly readable.
Under rootless Podman, files written into that directory land as admin on the host even though the process believes it is root, because of the UID mapping Part 3 set up. That is what makes tar in the backup script below work without sudo.
Requires= plus After= on the database unit means Postgres starts first and, combined with Notify=healthy above, is actually accepting connections before the app tries. On a reboot both come back in the right order without anything retrying.
Why a Unit and Not podman run
The version of this that most deploy pipelines start with — mine included — is a podman run -d in the SSH step, preceded by podman stop and podman rm:
podman stop umimaclean || true
podman rm umimaclean || true
podman run -d --name umimaclean --restart unless-stopped ...It works, and it is a reasonable place to start. Three things eventually pushed me off it.
--restart unless-stopped is not a boot policy. It restarts the container if it crashes while Podman is watching. After a reboot, nothing starts it — the flag records an intention that only podman-restart.service acts on, and only if you enabled that. A container that survives every crash and no reboots is a trap you spring on yourself once.
There is no dependency ordering. podman run cannot express “after Postgres is healthy”. The unit can, and on a cold boot that is the difference between the app coming up and the app crash-looping until Postgres finishes initialising.
The container definition lives in the pipeline. Every flag — network, mounts, env file — is in a YAML file on GitHub. Change one, and the only way to apply it is a deploy. Worse, the state of the server is now whatever the last successful workflow run happened to pass, which you cannot read from the server itself. With Quadlet, ~/.config/containers/systemd/ is the answer to “how is this deployed”, it is in the backup tarball, and the deploy step’s job shrinks to restart.
The directory mounted at /app/storage is the one holding customer photos. It is the only application state that is not in Postgres, and the only reason the runtime stage flattens build/ into /app rather than leaving it nested — the path has to match what app.makePath('storage') resolves to.
Start the database now. The application unit will not start yet — the image it names does not exist until the first pipeline run has pushed it.
systemctl --user daemon-reload
systemctl --user start umimaclean-db
systemctl --user status umimaclean-dbMigrations, and Who Runs Them
The application has no tables yet, and the rate_limits table it needs is created by the same migration run as everything else — so the very first request to any route fails without it, not just a rate-limited one.
The command is:
node ace migration:run --force--force is required in production. Adonis makes you type it so that “run migrations” is never something that happens because a script was reused in the wrong place.
Three decisions here, and they are the ones that make or break the pipeline design.
Migrations do not run at boot. Nothing in bin/server.ts or the providers triggers them. That is correct: with one container it would be redundant, and the day there are two it would be two processes racing to alter the same table. It also means the pipeline is responsible for running them, explicitly, as a step you can watch fail.
They run against the new image, before the restart. The new code expects the new schema. Restarting first leaves new code talking to an old schema for however long the migration takes — which on the day you add a NOT NULL column is long enough for real requests to hit it.
They run in a throwaway container, not in the live one. podman exec umimaclean node ace migration:run would work, but only against the currently running — that is, old — image. A podman run --rm from the freshly pulled image is the version that matches the code about to be deployed.
Seeding is the one thing that stays manual, because it happens once:
podman run --rm --network apps \
--env-file ~/apps/umimaclean/.env \
ghcr.io/r3p-dev/skripsi:master \
node ace db:seeddatabase/seeders/01_service_seeder.ts creates the service and price rows. 02_order_seeder.ts is development data — read index_seeder.ts before pointing this at a database with real orders in it. This is emphatically not something to put in a pipeline that runs on every push.
To see where a database actually stands:
podman exec umimaclean node ace migration:statusCaddy
umimaclean.example.com {
encode zstd gzip
reverse_proxy umimaclean:3333
}podman exec caddy caddy reload --config /etc/caddy/CaddyfileThat is genuinely all of it for the common case, but three application-specific details are worth knowing about before they surprise you.
Server-sent events. Caddy detects text/event-stream responses and flushes them immediately, so the realtime channels from Part 6 work without configuration. If you want it stated rather than inferred, scope it to the Transmit endpoints — confirm the exact paths with node ace list:routes:
umimaclean.example.com {
encode zstd gzip
reverse_proxy /__transmit/* umimaclean:3333 {
flush_interval -1
}
reverse_proxy umimaclean:3333
}Note also that config/transmit.ts sets pingInterval: false, so an idle SSE connection sends nothing at all. Caddy does not time out proxied responses by default, so this is fine here — but any other proxy or corporate middlebox in the path may quietly close a stream that has been silent for a few minutes. The Transmit client reconnects, so the symptom is not a broken page, it is updates arriving late.
Uploads are not static files. serveFiles: true in config/drive.ts means /uploads/* is handled by the application, deliberately, because those files are private and access is checked per request. Do not be tempted to point Caddy at the storage directory directly to “save a hop” — that removes the authorisation.
Body size. Caddy sets no request body limit by default, so the effective limit is whatever config/bodyparser.ts allows for multipart uploads. If you add request_body { max_size ... } in Caddy, keep the two numbers in agreement, or a photo that Adonis would have accepted gets rejected by the proxy with an error the frontend cannot explain.
The Proxy Detail That Breaks Rate Limiting
This one is specific enough to deserve its own section, because nothing about it looks broken.
start/limiter.ts keys most of its limiters on ctx.request.ip():
export const loginLimiter = limiter.define('auth-login', (ctx) => {
const phone = String(ctx.request.input('phone', 'guest'))
return limiter
.allowRequests(5)
.every('1 minute')
.blockFor('5 minutes')
.usingKey(`${ctx.request.ip()}:${phone}`)
})Behind a reverse proxy, every request arrives from Caddy. The real client address is in X-Forwarded-For, which Caddy sets automatically — but Adonis only believes that header when the immediate peer is a trusted proxy, and it trusts loopback by default. Caddy is not on loopback here; it is another container on the apps network with an address like 10.89.0.4.
So the untrusted-proxy behaviour is what you get: request.ip() returns Caddy’s container address, identically, for every visitor on the internet. The login limiter still works — five attempts a minute, per phone number — but the registration limiter now has one shared bucket for the entire world. Ten signups in a minute from anywhere locks out everyone.
Check it before you trust it. Log request.ip() on any route and hit it from your phone on mobile data:
journalctl --user -u umimaclean -fIf what comes back is a 10.x address, add the trusted proxy configuration to config/app.ts:
import proxyAddr from 'proxy-addr'
export const http = defineConfig({
trustProxy: proxyAddr.compile(['loopback', 'linklocal', 'uniquelocal'])
// ...the rest unchanged
})uniquelocal covers the private ranges Podman assigns, so the forwarded header is honoured from Caddy and ignored from anywhere else. This is also what makes request.protocol() report https rather than http inside the container.
One More Trap, For Later
config/shield.ts has CSRF enabled with exceptRoutes: [], and its own comment says the list is there for external webhooks.
The Midtrans notification endpoint described in Part 8 is exactly that. Midtrans posts to it from its own servers with no session and no CSRF token, so it must be listed there, and the SHA-512 signature check is what secures it instead. This is not a problem in the repository as it stands — that route is not registered yet — but it will fail the first time it is, and it fails in a way that points nowhere useful: payments simply never confirm, no error appears in the customer’s browser, and the gateway dashboard shows a 403 you have to go looking for.
Set the notification URL in the Midtrans dashboard to the deployed path and test it with a sandbox transaction before trusting production traffic to it.
The Server Side of the Deploy
Before any workflow YAML, the server needs a script that knows how to deploy itself. Everything CI does remotely is a single call to this file, which keeps the procedure on the machine it applies to rather than scattered across a YAML that only GitHub can read.
nano ~/apps/umimaclean/deploy.sh
chmod +x ~/apps/umimaclean/deploy.sh#!/usr/bin/env bash
set -euo pipefail
APP_DIR="/home/admin/apps/umimaclean"
IMAGE="${1:-ghcr.io/r3p-dev/skripsi:master}"
export XDG_RUNTIME_DIR="/run/user/$(id -u)"
case "$IMAGE" in
ghcr.io/r3p-dev/skripsi:*) ;;
*)
echo "refusing to deploy unexpected image: $IMAGE" >&2
exit 1
;;
esac
podman pull "$IMAGE"
podman run --rm \
--network apps \
--env-file "$APP_DIR/.env" \
"$IMAGE" \
node ace migration:run --force
podman tag "$IMAGE" ghcr.io/r3p-dev/skripsi:master
systemctl --user restart umimaclean
for _ in $(seq 1 15); do
if podman exec umimaclean wget -qO- http://127.0.0.1:3333/ >/dev/null 2>&1; then
echo "deployed: $IMAGE"
podman image prune -f
exit 0
fi
sleep 2
done
echo "app did not answer after restart" >&2
systemctl --user status umimaclean --no-pager >&2
exit 1Four lines in there are the ones I would not have written on the first attempt.
export XDG_RUNTIME_DIR. A non-interactive SSH command does not necessarily get one, and without it systemctl --user fails with Failed to connect to bus. This is the single most common way a deploy pipeline dies on a rootless Podman host, and the error tells you nothing about SSH. Lingering has to be enabled too — Part 3 covers that.
The case guard. The workflow passes the image tag as an argument over SSH. Validating that it points at this repository’s package means a compromised workflow token cannot talk the server into running an arbitrary image from an arbitrary registry.
podman tag. The pipeline deploys an immutable commit-SHA tag so it is unambiguous which commit is live. Retagging it as :master afterwards keeps the Quadlet unit’s Image= line honest, so a reboot brings back the version that was actually deployed rather than whatever :master last happened to be pulled.
The health loop. systemctl --user restart returns as soon as the container starts, not when the app is answering. Without the loop, a container that boots and immediately dies on an env validation error is a green pipeline. Fifteen tries, two seconds apart, is enough time for Adonis to boot and connect to Postgres, and dumps the unit status into the workflow log if it does not.
set -euo pipefail means a failed migration exits before the restart. The old container keeps serving, and the pipeline goes red. That is the behaviour you want at 11pm.
There are a few seconds of downtime during the restart. Open SSE connections drop and the Transmit client reconnects on its own. For one shop with a handful of staff, that is not worth engineering around.
The Deploy Key
GitHub needs a way in. Not your key — a dedicated one, generated on the server, used for nothing else.
ssh-keygen -t ed25519 -C "github-actions-deploy" -f ~/.ssh/gha_deploy -N ""
cat ~/.ssh/gha_deploy.pub >> ~/.ssh/authorized_keysThen, in the repository’s Settings → Secrets and variables → Actions:
VPS_SSH_KEY— the contents of~/.ssh/gha_deploy, the private halfVPS_HOST— the server’s addressVPS_PORT— the SSH port, which Part 1 moved off 22VPS_USER—adminVPS_FINGERPRINT— the server’s host key fingerprint, obtained below
Delete ~/.ssh/gha_deploy from the server once the private half is in GitHub. The server only needs the public half.
The fingerprint is the one people skip, and it is worth thirty seconds. Run this somewhere you trust — ideally on the server itself:
ssh-keyscan -p 2222 -t ed25519 your.server | ssh-keygen -lf -Take the SHA256:... field. Without it, appleboy/ssh-action connects without verifying the host key at all, which means every deploy trusts whatever host answers at that address. The whole point of a deploy pipeline is that nobody is watching it — that is exactly when you want the connection to be authenticated in both directions.
If you want to go further, restrict the key to a forced command in ~/.ssh/authorized_keys:
command="/home/admin/apps/umimaclean/deploy.sh $SSH_ORIGINAL_COMMAND",no-agent-forwarding,no-port-forwarding,no-pty,no-X11-forwarding ssh-ed25519 AAAA... github-actions-deployA key with a forced command cannot open a shell, so leaked secrets get “somebody can deploy a build of this repository” rather than “somebody has a shell on the server” — and it is why the case guard in deploy.sh is doing real work rather than decorating. The catch is that appleboy/ssh-action sends a multi-line script and expects a shell to run it, so this variant means dropping the action and calling ssh directly with the image tag as the entire remote command. I run the forced-command version; the workflow below shows the action, because that is the shape most people start from.
The Workflow
One file, three jobs, on master. Note the branch name explicitly — this repository predates GitHub’s switch to main as the default, and a workflow watching a branch that does not exist is a pipeline that never runs and never complains about it.
mkdir -p .github/workflows
nano .github/workflows/ci.ymlname: CI
on:
push:
branches: [master]
pull_request:
branches: [master]
concurrency:
group: deploy-master
cancel-in-progress: false
permissions:
contents: read
packages: write
env:
IMAGE_NAME: ghcr.io/${{ github.repository }}
jobs:
test:
runs-on: ubuntu-latest
services:
postgres:
image: postgres:18-alpine
env:
POSTGRES_USER: umimaclean
POSTGRES_PASSWORD: umimaclean
POSTGRES_DB: umimaclean_test
ports: ['5432:5432']
options: >-
--health-cmd "pg_isready -U umimaclean"
--health-interval 10s
--health-timeout 5s
--health-retries 5
env:
TZ: UTC
PORT: 3333
HOST: 127.0.0.1
NODE_ENV: test
LOG_LEVEL: warn
APP_KEY: TestKeyNotASecret000000000000000
APP_URL: http://127.0.0.1:3333
SESSION_DRIVER: memory
DRIVE_DISK: fs
LIMITER_STORE: memory
DB_HOST: 127.0.0.1
DB_PORT: 5432
DB_USER: umimaclean
DB_PASSWORD: umimaclean
DB_DATABASE: umimaclean_test
MIDTRANS_MERCHANT_ID: test
MIDTRANS_SERVER_KEY: test
FONNTE_API_KEY: test
steps:
- uses: actions/checkout@v6
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v6
with:
node-version: 24
cache: pnpm
- run: pnpm install --frozen-lockfile
- run: pnpm lint
- run: pnpm typecheck
- run: node ace migration:run --force
- run: node ace test unit functionalYAML, unlike everything else in this post, must be indented with spaces. A tab anywhere in that file is a parse error, and GitHub reports it as a workflow that simply does not appear.
IMAGE_NAME derives from github.repository, which makes the image ghcr.io/r3p-dev/skripsi — the repository is the thesis project, the application inside it is UmimaClean. Deriving it rather than hardcoding a nicer name is worth it: GITHUB_TOKEN can push to a package under the repository’s own path without any additional permissions, and a package named after something else needs its access configured by hand before the first push works.
concurrency with cancel-in-progress: false is the one setting people get backwards. Two pushes in quick succession must not produce two deploys running at once, but cancelling a deploy that is halfway through a migration is worse than queueing. Queue them.
The test job’s environment block is .env.example with test values, and every variable in start/env.ts has to appear or the app will not boot far enough to run a single test. APP_KEY here is a literal, not a secret, and should be — it signs nothing that outlives the job, and putting it in secrets only makes the workflow harder to read for people who then wonder what it protects.
SESSION_DRIVER=memory matches .env.test. LIMITER_STORE=memory keeps rate limiter state out of the test database, which matters because the functional tests call testUtils.db().truncate() between tests and a limiter counting in Postgres would be truncated mid-suite.
node ace migration:run --force before the tests, because truncate() empties tables — it does not create them.
On suite selection. node ace test unit functional runs the two suites that have files. adonisrc.ts also declares a browser suite, and tests/bootstrap.ts wires up browserClient. Playwright’s npm package comes in as a peer dependency, but its browser binaries do not — the day the first tests/browser spec lands, this needs a playwright install --with-deps chromium step before it, and the failure until then reads as a missing executable rather than a missing CI step.
Build and Deploy
The other two jobs, in the same file:
build:
needs: test
if: github.event_name == 'push' && github.ref == 'refs/heads/master'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Login to GHCR
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | \
podman login ghcr.io \
-u ${{ github.actor }} \
--password-stdin
- name: Build image
run: |
podman build \
--ulimit nofile=65535:65535 \
--layers \
--cache-from $IMAGE_NAME:buildcache \
--cache-to $IMAGE_NAME:buildcache \
-t $IMAGE_NAME:${{ github.sha }} \
-t $IMAGE_NAME:master \
.
- name: Push image
run: |
podman push $IMAGE_NAME:${{ github.sha }}
podman push $IMAGE_NAME:master
deploy:
needs: build
runs-on: ubuntu-latest
environment: production
steps:
- name: Deploy to VPS
uses: appleboy/ssh-action@v1
with:
host: ${{ secrets.VPS_HOST }}
port: ${{ secrets.VPS_PORT }}
username: ${{ secrets.VPS_USER }}
key: ${{ secrets.VPS_SSH_KEY }}
fingerprint: ${{ secrets.VPS_FINGERPRINT }}
script: |
~/apps/umimaclean/deploy.sh ${{ env.IMAGE_NAME }}:${{ github.sha }}Some notes on why this is shaped the way it is.
Podman on the runner, not Docker. ubuntu-latest ships both. Building with the same tool that runs the image means one mental model instead of two, and podman build reads the Containerfile in the working directory without being told where it is. The output is an ordinary OCI image either way — nothing downstream can tell which tool assembled it.
--ulimit nofile=65535:65535. pnpm opens a great many file descriptors while linking a dependency tree this size, and the runner’s default soft limit is low enough to produce EMFILE partway through an install. The error names a random package, which sends you looking in exactly the wrong place.
--layers with a registry cache. This is the one thing missing from a plain podman build in CI. The --mount=type=cache in the Containerfile is a build-host cache — on a hosted runner the host is destroyed when the job ends, so a pnpm store cache that never survives is a pnpm store cache that never helps. --cache-to/--cache-from against a :buildcache tag pushes the intermediate layers to GHCR, so the next run’s pnpm fetch layer is a registry pull rather than a cold install. Without it, every push re-fetches every dependency and recompiles the entire React frontend, and the honest question to ask is whether that is worth optimising at all: a five-minute build you do twice a week is fine. Add the cache when the wait starts changing your behaviour.
Two tags, one build. ${{ github.sha }} is immutable and is what gets deployed. :master is the moving pointer the Quadlet unit names, so a reboot pulls the same commit that was last deployed. Passing the SHA tag to deploy.sh means the workflow log, the registry, and podman ps all agree on which commit is running — :latest cannot tell you that.
environment: production is optional but cheap. Adding a required reviewer to that environment in repository settings turns every deploy into a one-click approval, which is worth having the first few weeks and easy to remove later.
A note on the runner’s architecture. GitHub’s hosted runners are x86-64. If your VPS is ARM, the image built here will not run on it, and the error — exec format error — arrives at container start rather than at build. --platform linux/arm64 with qemu-user-static fixes it, at the cost of a build slow enough that a self-hosted ARM runner starts to look reasonable.
The server has to be able to pull what the runner pushed. A package published to GHCR is private by default even when the repository is public, and the failure is on the server, hours later, as unauthorized from podman pull. Either make the package public in its GitHub settings — reasonable for a public repository, and it means the VPS stores no registry credentials at all — or log in once on the server with a read-only classic token scoped to read:packages:
podman login ghcr.io -u r3p-dev --authfile ~/.config/containers/auth.jsonand point the systemd unit at that file so a reboot can pull too, by adding one line under [Service] in umimaclean.container:
Environment=REGISTRY_AUTH_FILE=%h/.config/containers/auth.jsonThe default auth location is under $XDG_RUNTIME_DIR, which is cleared on reboot — which is exactly when you need it and are least likely to be watching.
The one real benefit of moving the build off the server, incidentally, is that it no longer has to fit. A Vite build of this frontend on a 1 GB VPS is exactly the workload that gets OOM-killed with an error mentioning nothing about memory. On a runner with 16 GB it is a non-event.
Rollback
Every deployed image is still in the registry under its commit SHA, and deploy.sh takes an image argument. So a rollback is the same operation with an older tag. Find the commit you want — git log --oneline is enough, since the tag is the full SHA — then, on the server:
~/apps/umimaclean/deploy.sh \
ghcr.io/r3p-dev/skripsi:9f2c1abd4e7b3c0518a6f21d9ce4470b8a3d5e62This is the argument for tagging with ${{ github.sha }} rather than :latest. A registry full of :latest can tell you what is newest and nothing else; a registry tagged by commit means every deploy this project has ever made is still addressable, and rolling back is choosing one of them rather than rebuilding an old commit and hoping the result is identical.
With one caveat that is easy to say and easy to forget: this rolls back code, not schema. deploy.sh runs migration:run on the way in, and running it against an older image is a no-op — it does not undo anything. If the deploy you are backing out of dropped a column, the old code is now talking to a schema that no longer has it, and the honest fix is a forward migration plus a new deploy. Rollback is a fast path for “the new code is wrong”, not for “the new migration was wrong”.
Backups
Two things are irreplaceable: the database and the uploads directory. Everything else — the image, the units, the source — can be rebuilt from the registry or from GitHub.
nano ~/apps/umimaclean/backup.sh#!/usr/bin/env bash
set -euo pipefail
APP_DIR="/home/admin/apps/umimaclean"
DEST="/home/admin/backups/umimaclean"
STAMP=$(date +%F)
mkdir -p "$DEST"
podman exec umimaclean-db \
pg_dump -U umimaclean -d umimaclean --format=custom \
> "$DEST/db-$STAMP.dump"
tar czf "$DEST/storage-$STAMP.tar.gz" -C "$APP_DIR" storage
find "$DEST" -type f -mtime +14 -deleteThe uploads half is an ordinary tar of an ordinary directory, which is the practical argument for the bind mount over a named volume. Backing up a named volume means podman volume export or a throwaway container with the volume attached — neither hard, both one more thing that has to be right in a script nobody reads until the day it matters.
Run it nightly with a user timer:
systemctl --user edit --force --full umimaclean-backup.timerA backup you have never restored is a hypothesis. Test it into a scratch database:
podman exec -i umimaclean-db \
pg_restore -U umimaclean -d umimaclean_restore_test --clean --if-exists \
< ~/backups/umimaclean/db-2026-08-20.dumpAnd get the dumps off the machine. A backup living on the same VPS as the database protects you against exactly one failure mode — the one where you delete rows by hand — and none of the others.
Things That Went Wrong
Container restarted every few seconds, no useful log. Env validation. journalctl --user -u umimaclean -n 50 has the actual message, and it names the variable. Usually MIDTRANS_SERVER_KEY or FONNTE_API_KEY, empty because that part is not wired up yet.
connect ECONNREFUSED on first boot. Postgres still initialising. Notify=healthy on the database unit fixes it properly; restarting the app by hand only fixes it once.
502 from Caddy, container healthy. HOST left at localhost. The app is listening, just not on an interface anything outside its namespace can reach.
pnpm install --frozen-lockfile failing in the build stage. Either Corepack is not enabled and pnpm is the wrong version, or pnpm-workspace.yaml was not copied and the overrides no longer match the lockfile. This one fails identically on a runner and on a laptop, which is at least honest of it.
EMFILE: too many open files during install. The runner’s default descriptor limit. --ulimit nofile=65535:65535 on the build command; the error names whichever package happened to be linking at the time, which is never the problem.
ERR_PACKAGE_IMPORT_NOT_DEFINED on the first request. package.json missing from the production stage, so Node cannot resolve the #controllers/* subpath imports. The container starts fine — it fails when a route is hit.
The app came back after a crash but not after a reboot. --restart unless-stopped on an ad-hoc podman run, with no podman-restart.service enabled. A Quadlet unit with WantedBy=default.target and lingering is the version that survives both.
Deployed successfully, then 500s everywhere. A pipeline with no migration step. This is the failure mode of the simple stop/rm/run deploy: it is completely correct about containers and knows nothing about schemas.
Failed to connect to bus in the deploy job. No XDG_RUNTIME_DIR in a non-interactive SSH session, or lingering not enabled for admin. The deploy script exports it; if you call systemctl --user from the workflow directly instead, you get to discover this yourself.
The workflow never ran. Watching main on a repository whose default branch is master. GitHub does not warn you about this — there is simply no run, and the commit gets no status check at all.
unauthorized pulling from GHCR on the server. Private package, no podman login. The runner pushing successfully tells you nothing about whether the VPS can pull.
exec format error on container start. An x86-64 image on an ARM VPS. Built fine, pushed fine, cannot execute.
Green pipeline, dead site. A deploy step that ends at systemctl --user restart reports success the moment the container starts, including when it starts and immediately exits. That is what the health loop in deploy.sh is for.
TZ set but dates still shifted. No tzdata in the Alpine image.
Uploads gone after a deploy. Nothing mounted at /app/storage, or mounted at a path that does not match where app.makePath('storage') resolves. Check with podman exec umimaclean ls -la /app/storage while the app is running.
Rate limiter blocking everyone at once. The trustProxy issue above. Nothing about it looks like a proxy problem until you print the IP.
Build killed with no error. Out of memory — the failure from back when the server built its own images. A Vite build of a React app this size is comfortably enough to OOM a 1 GB VPS, and the reason lives only in dmesg | grep -i "killed process". Moving the build to a runner is what actually fixed this; the swap file from Part 2 is what made it survivable before that.
The same env file behaving differently in two places. ~/apps/umimaclean/.env is read by systemd as EnvironmentFile= and by Podman as --env-file, and the two parsers do not agree on quoting — systemd strips matched quotes, Podman keeps them. Keep every value unquoted and free of #, and the file means the same thing to both. A quoted DB_PASSWORD that works under systemd and fails during the migration step is a genuinely confusing hour.
Closing
Once it is running, the shape of the thing is small enough to describe in a sentence: two containers on a private network, a volume for Postgres and a directory for uploads, one env file, one Caddyfile entry, one workflow, and a thirty-line script on the server that pulls, migrates, restarts, and checks.
The split between those last two is the part I would keep in any future project. The workflow decides when to deploy and what — a branch, a test suite, an image tag. The server decides how, and that knowledge lives on the machine it describes, where I can run it by hand at 11pm without a GitHub outage being part of the story. Pipelines that encode the entire procedure in YAML are pipelines you cannot use when you most need to.
What took the time was none of the architecture. It was HOST=localhost, a missing timezone database, a storage path off by one directory, a missing XDG_RUNTIME_DIR, and an IP address that was technically correct and completely useless. Every one of them presented as an application bug and was fixed in infrastructure.
Which is the argument for writing the list down. The next deploy of the next app will hit four of these again, and the second time through they are ten minutes instead of an evening.
The application itself is documented across the stack series, the product reasoning in Building an Operations Platform for a Shoe Cleaning Shop, and the server underneath it in Setting Up a VPS, Part 1. Source at github.com/r3p-dev/skripsi.