Yellow Jack with Harbor

From docs/integrations/registry-harbor.md at v0.1.0 ·View the source on GitHub

This guide connects Yellow Jack to Harbor, for container images. It covers both places the gate can sit: behind Harbor, as the upstream of a Harbor proxy-cache project, and in front of Harbor, as the address your clients pull from.

Tested. e2e/registry_harbor.sh installs Harbor with Harbor’s own installer, runs the files this guide uses (docs/integrations/harbor/), and pulls with a real Docker daemon. It checks that each pull reached the gate through Harbor (behind) or reached Harbor through the gate (in front), not only that the pull was refused. It also checks the cache behaviour described below, and that the same pulls succeed when the deny entry is removed. Last run: 2026-10-05.


Harbor	v2.15.2, online installer `harbor-online-installer-v2.15.2.tgz` (sha256 `88f6a7436b31890e8e472972a7433d36b7d6a36de9adeb86337fdc9fe7fb5fa3`)
Harbor images	`goharbor/*:v2.15.2`; the digests are pinned in `e2e/registry_harbor.sh`
Docker host	`docker:27-dind@sha256:aa3df78ecf320f5fafdce71c659f1629e96e9de0968305fe1de670e0ca9176ce` (Docker Engine 27.5.1)
Files	`docs/integrations/harbor/`

Not run in CI. The test needs a privileged container and pulls about 1 GB of Harbor images each time. Run it locally with sh scripts/dev.sh harbor.

Which position to choose

The two positions stop different things.

Harbor behind the gate. Clients pull from a Harbor proxy-cache project. The project’s registry endpoint is the gate. Nothing changes on a developer machine except the project name in the image reference.

The gate sees every pull Harbor does not answer from its cache. A refused image never enters Harbor.
An image Harbor has cached is still served after you deny it. By digest, Harbor does not ask the gate. By tag, Harbor asks the gate, gets the refusal, and serves its own copy anyway. To stop both, delete the cached copy and run garbage collection (see Withdraw a cached image).
The client sees Harbor’s error, not the gate’s reason: repository ... not found.

Gate in front of Harbor. Clients pull from the gate. The gate’s upstream is Harbor.

The gate sees every pull, by tag or by digest, cached or not. A deny applies at once.
The client sees the gate’s reason.
Deny entries name the Harbor project too, for example hub/library/busybox, because that is the path clients ask for.
Clients must not have another route to Harbor.

Neither position covers a client that pulls straight from Docker Hub.

1. Install Harbor

Install Harbor with its own installer, as its documentation describes. The test used the online installer for v2.15.2. harbor-yml.sh writes the harbor.yml it used, with the passwords taken from your environment:

export YJ_HARBOR_ADMIN_PASSWORD="$(openssl rand -hex 12)"
export YJ_HARBOR_DB_PASSWORD="$(openssl rand -hex 12)"
tar xzf harbor-online-installer-v2.15.2.tgz
sh docs/integrations/harbor/harbor-yml.sh > harbor/harbor.yml
cd harbor && ./install.sh

✔ ----Harbor has been installed and started successfully.----

The file differs from Harbor’s harbor.yml.tmpl in one block:

hostname: harbor.test
http:
  port: 8080

and has no https: block. harbor.test is the trial’s name for Harbor. Harbor writes it into the token URL it gives clients, so clients, and a gate in front, must be able to resolve it. Use your real DNS name, and TLS, in a deployment.

Why the test runs Harbor inside a container. The installer’s prepare step mounts host paths (/, /data) and expects a Linux host. The test runs it inside a docker:dind container, which is a clean Linux Docker host on any machine. The gates, the approval service and every docker pull below run in that same daemon. That daemon resolves harbor.test to itself through /etc/hosts, and is started with --insecure-registry harbor.test:8080 because the trial is plain HTTP.

2. Start the gates

Build the gate and approval images from this repository on the Harbor host, then start them beside Harbor:

docker build -t yellowjack-integrations-firewall .
docker build -t yellowjack-integrations-approval -f approval/Dockerfile .
cd docs/integrations/harbor && sh start-gates.sh

start-gates.sh joins all three to harbor_harbor, the network Harbor’s installer creates, so Harbor reaches the gate by name.

The gate behind Harbor (gate-oci). It publishes no port. Its upstream is Docker Hub, the default, and FW_PUBLIC_URL is the address Harbor uses to reach it:

FW_ECOSYSTEM=oci
# FW_UPSTREAM is left at its default, https://registry-1.docker.io
FW_PUBLIC_URL=http://gate-oci:8080
FW_SCORECARD_MODE=off
FW_ALLOW_LIST=/lists/allow.txt
FW_DENY_LIST=/lists/deny.txt
FW_APPROVAL_URL=http://approval-yj:8090

Its deny list (lists/behind/deny.txt) has library/busybox, as Docker Hub names it. Harbor removes its project name before it asks the gate.

The gate in front of Harbor (front-oci), on port 8380. Its upstream is Harbor, and FW_PUBLIC_URL is the address clients use:

FW_LISTEN_ADDR=:8380
FW_ECOSYSTEM=oci
FW_UPSTREAM=http://harbor.test:8080
FW_PUBLIC_URL=http://localhost:8380

with the same last four lines as gate-oci. Its deny list (lists/front/deny.txt) has hub/library/busybox. In the trial, harbor.test is not in DNS, so start-gates.sh adds a host entry for it when you set HARBOR_IP to the address of Harbor’s nginx container.

3. Configure Harbor

configure.sh makes every change through the Harbor REST API:

HARBOR_PASSWORD="$YJ_HARBOR_ADMIN_PASSWORD" sh docs/integrations/harbor/configure.sh

1. registry endpoint yellowjack-gate (http://gate-oci:8080)
   created
2. registry endpoint docker-hub (https://hub.docker.com)
   created
3. proxy-cache project gated (endpoint 1)
   created
4. proxy-cache project hub (endpoint 2)
   created

It is safe to run twice. The calls, with -u admin:<password> and API=http://harbor.test:8080/api/v2.0:

Step	Call	Body
1	`POST $API/registries`	`{"name":"yellowjack-gate","type":"docker-registry","url":"http://gate-oci:8080","insecure":true}`
2	`POST $API/registries`	`{"name":"docker-hub","type":"docker-hub","url":"https://hub.docker.com"}`
3	`POST $API/projects`	`{"project_name":"gated","registry_id":<id of yellowjack-gate>,"metadata":{"public":"true"}}`
4	`POST $API/projects`	`{"project_name":"hub","registry_id":<id of docker-hub>,"metadata":{"public":"true"}}`

GET $API/registries?q=name%3Dyellowjack-gate returns the id. In the UI the same objects are under Administration, Registries (endpoint type “Docker Registry” for the gate), and Projects, New Project, with Proxy Cache on.

Both projects are public, so the pulls below need no docker login.

4. Harbor behind the gate

docker pull harbor.test:8080/gated/library/alpine:3.20

Status: Downloaded newer image for harbor.test:8080/gated/library/alpine:3.20

docker pull harbor.test:8080/gated/library/busybox:latest

Error response from daemon: unknown: repository gated/library/busybox not found

Harbor turned the gate’s 403 into “not found”. Harbor’s core log has the status:

[ERROR] [/server/middleware/repoproxy/proxy.go:162]: failed to proxy manifest, fallback to local, request uri: /v2/gated/library/busybox/manifests/latest, error: http status code: 403, body:

and the gate’s log has the reason:

HEAD [verdict-blocked] library/busybox:latest -> allowed=false score=0.0 hasScore=false (package "library/busybox:latest" is on this organisation's deny list; ...)

What the gate sees. The approval service records each verdict with the address the request came from. Behind Harbor that is the harbor-core container. The test reads it with curl -s "http://localhost:8390/v1/events?package=library/busybox:latest&ecosystem=oci".

The same daemon pulling docker.io/library/busybox:latest straight from Docker Hub gets the image. The gate only covers pulls that go through it.

Withdraw a cached image

This was measured with alpine:3.20, after the pull above.

What Harbor stores, and when. Within seconds of the pull, Harbor stored the platform manifest the client fetched (linux/amd64). It stored the tag 3.20 later, pointing at an index Harbor built from the platforms it holds: between 75 seconds and six and a half minutes later across the test runs. Until then, a pull by tag still went to the gate.

Then, with the tag stored:

Add library/alpine to the deny list.
Pull by tag: served. Harbor asked the gate, was refused, and served its stored copy. Its core log says failed to proxy manifest, fallback to local, request uri: /v2/gated/library/alpine/manifests/3.20, error: http status code: 403.
Pull by the stored platform digest (gated/library/alpine@sha256:...): served. The gate had no record of the request.
Delete the repository from the project, then run garbage collection (withdraw.sh). Now a pull by tag is refused (Error response from daemon: unknown: repository gated/library/alpine not found), and a pull by digest reaches the gate and is refused.

HARBOR_PASSWORD=... sh docs/integrations/harbor/withdraw.sh gated library/alpine

deleted repository gated/library/alpine
garbage collection finished

withdraw.sh calls DELETE $API/projects/gated/repositories/library%252Falpine (the slash is encoded twice), then POST $API/system/gc/schedule with {"schedule":{"type":"Manual"},"parameters":{"delete_untagged":true,"workers":1}}, and waits for that job. Garbage collection runs across the whole Harbor instance.

Check with a pull by digest afterwards. Deleting the repository is not enough on its own: in five trials by hand, a pull by digest straight after the delete was still served in four. After garbage collection it was refused, once only on a second try about a minute later. The test polls for up to five minutes; in its runs the first try was refused.

Check from a clean client. A Docker daemon that already holds the image answers a pull by digest from its own store and asks no registry at all, in either position. Remove the image locally first (docker rmi) or check from another machine.

5. Gate in front of Harbor

docker pull localhost:8380/hub/library/alpine:3.20

Status: Downloaded newer image for localhost:8380/hub/library/alpine:3.20

docker pull localhost:8380/hub/library/busybox:latest

Error response from daemon: pull access denied for localhost:8380/hub/library/busybox, repository does not exist or may require 'docker login': denied: blocked by firewall: package "hub/library/busybox:latest" is on this organisation's deny list; refused without contacting upstream. This is a local policy decision, not a published malware advisory -- ask whoever maintains the firewall's deny list. ...

The first half of that message is Docker’s guess, and it is wrong. The text after denied: is the gate’s. This daemon (Docker Engine 27, graphdriver image store) fetches the manifest with GET and prints the body. A daemon on the containerd image store resolves the tag with HEAD and prints only 403 Forbidden.

What Harbor sees. Harbor’s proxy log (/var/log/harbor/proxy.log on the Harbor host) shows the alpine pull coming from the gate’s address, and nothing for busybox:

172.19.0.12 - "HEAD /v2/hub/library/alpine/manifests/3.20 HTTP/1.1" 200 0 "-" "docker/27.5.1 ..."

172.19.0.12 was the front-oci container on that run.

A cached image is refused as soon as it is denied. This was measured with hub/library/alpine, cached by the pull above: after the deny entry, the next pull failed with the gate’s reason, by tag and by digest. No Harbor step is needed.

Both projects are public, so Harbor hands out tokens for anonymous pulls, and the pull through the gate worked without any credential on the gate. For a private project, note how the gate handles credentials: it relays the client’s own Authorization header, and FW_UPSTREAM_AUTH or FW_UPSTREAM_AUTH_FILE covers only the gate’s own probe requests (see CONFIGURATION.md). Do not put a user name and password in the FW_UPSTREAM URL: the gate writes that URL to its log. A private project was not tested here.

Common failures

The pull through Harbor says not found for an image that exists. The gate refused it. Read the gate’s log or the approval service’s records for the reason.

A denied image still pulls through Harbor. Harbor had it cached. By tag, Harbor serves its copy when the gate refuses; by digest, it does not ask. See Withdraw a cached image.

A deny entry in front of Harbor does not match. It must include the project: hub/library/busybox, not library/busybox.

Moving from the trial to a deployment

Install Harbor on its own Linux host, with TLS and a real host name. Set hostname in harbor.yml to the name clients and the front gate both resolve.
Give the gate behind Harbor an address Harbor can reach, and put that address in the registry endpoint and in the gate’s FW_PUBLIC_URL.
Serve the gate in front of Harbor over TLS, set its FW_PUBLIC_URL to that address, and keep clients off Harbor’s own port.
Use a persistent approval database (APPROVAL_DATABASE_URL). The trial keeps audit records in memory.