Posted on August 02, 2026
Category: Technology
Tags: openstreetmap, fastapi, sqlite, docker, cloudflare-tunnel, unix-socket, python, golf, geospatial, gpx, kml, odbl, rate-limiting, llm
Views: 105
I wanted hole-by-hole GPS coordinates for golf courses. Tee boxes, greens, yardages — the kind of data a rangefinder app or a GPS watch needs. Commercial providers sell this behind per-call licensing. OpenStreetMap has a surprising amount of it for free, buried in a 12GB binary blob and tagged by thousands of volunteers who did not agree on a convention.
This is what it took to turn that into a public API, and the two things I got wrong along the way that only showed up in production-shaped environments.
The result is live at golfclubxy.willconia.com, running in Docker behind a Cloudflare Tunnel with no inbound TCP ports open at all.
OSM models a golf course as a leisure=golf_course polygon containing golf=hole ways, each of which may have par, handicap, ref, and a distance tag. May. The tags are optional and mappers use them inconsistently.
Some courses have every hole numbered with ref=1..18 and both a tee polygon and a green polygon. Others have eighteen unlabeled fairway lines and nothing else. A few have the course boundary and literally nothing inside it.
So the extractor classifies what it finds into tiers rather than pretending the data is uniform:
ref tags, high confidenceOf 12,719 US courses, 3,626 (28.5%) land in the weakest bucket. That number is the honest headline of the project. No amount of clever parsing fixes a course nobody mapped, and the API says so in a warnings array instead of quietly returning a plausible-looking guess.
A few of the judgement calls worth naming:
Tee box and green coordinates are bounding-box centers. OSM ships raw polygon vertices, not a center of any kind — the extractor computes one, using the same definition Overpass's own out center mode would (min/max lat and lon, averaged). For an irregular green this is not the pin position and not the centroid of the polygon — it is the center of the box that contains it. Close enough for a yardage readout, wrong if you expect surveying precision.
Practice greens get dropped. A driving range green tagged the same way as a real one will otherwise show up as hole 19.
Multiple tee colours per hole are kept as a ranked array rather than collapsed to one, because "white tee" means different things at different clubs.
Extraction runs offline against a PBF file and writes SQLite. The API never touches the PBF and never writes to the database — it is mounted read-only in the container.
Six endpoints, all read-only:
GET /health
GET /courses?q=&limit=&offset=
GET /courses/{osm_id}
GET /courses/{osm_id}/geojson
GET /courses/{osm_id}/gpx
GET /courses/{osm_id}/kml
The export formats were a deliberate choice. GPX is Topografix's open schema and KML has been an OGC standard since Google donated it in 2008 — neither carries format licensing. The coordinates are still OSM-derived, so ODbL attribution rides along inside the file:
<metadata>
<copyright author="© OpenStreetMap contributors">
<license>https://opendatacommons.org/licenses/odbl/1-0/</license>
</copyright>
</metadata>
That constraint decided something else for me. I also had a CSV layout that a consumer app expects — fixed columns, no room for a comment line, no metadata field. There is nowhere in that file to put the attribution. Adding a column breaks the format; a header comment breaks parsers that require the header on line one.
So the CSV converter exists as a local CLI flag, not an endpoint:
uv run python -m golfclubxy lookup "Mooresville Golf Club" --tee white \
--pins out/mooresvillegc.csv
ODbL obligations attach on distribution, not on local use. Exporting a file for myself triggers nothing. Serving that same file to third parties over HTTP does. The rule I settled on: a format gets an endpoint only if the license notice fits inside the format.
Public read-only data still needs a rate limiter. A single SQLite file means one client hammering it slows down everyone else through I/O contention. I added a fixed-window in-memory counter — 60 requests per IP per minute, no Redis, no external dependency.
It worked in every local test. Then I deployed it over a Unix domain socket and it silently stopped being a per-IP limit.
Starlette populates request.client from the socket peer address. Over a Unix socket there is no peer address — scope["client"] is None. Every request bucketed under the same fallback key, which turned "60 per IP" into "60 total, globally, shared by all users." The API would have started rejecting real traffic as soon as more than one person used it.
The fix is small; noticing was the hard part:
@staticmethod
def _client_key(request: Request) -> str:
if request.client is not None:
return request.client.host
forwarded = request.headers.get("cf-connecting-ip") or request.headers.get(
"x-forwarded-for", ""
)
return forwarded.split(",")[0].strip() or "unknown"
The ordering matters and is not cosmetic. Forwarded headers are trivially spoofable, so they are consulted only when there is no socket peer — that is, only when something like cloudflared is necessarily in front. If the app is ever exposed on a TCP port directly, request.client exists and the headers are ignored.
Two more things in the same area:
Pagination. /courses originally capped at 50 results with no way past them. With 12,719 courses, a query like "club" genuinely loses matches. Adding offset plus a has_more flag — computed by fetching limit + 1 rows — avoids a separate COUNT query entirely.
CORS is not access control. I opened it with allow_origins=["*"] and allow_credentials=False, and it is worth being precise about what that changes: nothing, for scrapers. curl and crawlers never look at CORS headers. It is a restriction browsers apply to protect their users. The same request a browser refused, curl had been happily receiving with a 200 the whole time. What actually changes is that third-party web apps become possible — and their traffic arrives from thousands of end-user IPs, which a per-IP limiter cannot aggregate.
One local-testing trap cost me an hour here. After adding CORS, my browser test still failed. The cause was not CORS at all — Chrome's Private Network Access blocks a public origin from reaching 127.0.0.1 independently of any CORS header. Testing it properly means putting both origins on 127.0.0.1 and varying only the port.
The deployment target was a small cloud box. Rather than the usual systemd + uvicorn + nginx stack, I used a Unix socket end to end:
[Cloudflare edge] --tunnel--> [cloudflared] --unix socket--> [docker: uvicorn]
nginx is not in that diagram, and does not need to be. The jobs a reverse proxy was doing here — read a socket, speak HTTP to the outside world, terminate TLS, own a public IP — are all handled by cloudflared and the Cloudflare edge. cloudflared's ingress config takes a unix: scheme directly:
ingress:
- hostname: golfclubxy.willconia.com
service: unix:/home/user/golfclubxy/run/app.sock
- service: http_status:404
The container publishes no ports. The host listens on no TCP port for this service. The only way in is the outbound tunnel the server itself established.
Some sharp edges, all of which cost me time:
The socket directory ownership. The image runs as UID 10001, but a bind-mounted host directory owned by UID 1000 is not writable by it, so sock.bind() fails with PermissionError: [Errno 13]. Changing a file's owner to a different UID needs CAP_CHOWN, so this is one of exactly two commands in the whole deployment that need root:
sudo chown 10001:10001 run
AF_UNIX paths have a length limit of roughly 107 bytes. I hit OSError: AF_UNIX path too long while testing from a deeply nested scratch directory. Do not install to a deep path.
rsync creates only the last component of a destination path. run/ is not the target of any transfer, so it never gets created — and chown on a missing directory fails with a confusing error. Also worth knowing: --delete, when the source is a list of files, treats the entire destination directory as fair game and will happily remove data/ and run/.
Never use --inplace or --append for the database. They overwrite the target in place, which means a torn, live-served database mid-transfer. rsync's default behavior — write a temp file, then rename — is atomic, and a running uvicorn keeps the old inode open until it restarts.
cloudflared service install installs a system unit and wants root. Running it as a user service instead means writing the unit yourself and enabling linger, or the tunnel dies the moment your SSH session ends:
sudo loginctl enable-linger $USER
And the failure that wasted the most time by far: three consecutive Docker builds failed on PyPI and registry timeouts, and the cause was my VPN. With the VPN up, the Docker daemon's egress was blocked while the host shell worked perfectly — which is exactly why it took three attempts to suspect. If your builds hang on downloads and everything else on the machine is fine, check the VPN before anything else.
Container overhead, since it comes up: median response 0.37ms containerized versus 0.39ms running directly on the host. There is no bridge networking to traverse (it is a Unix socket) and no overlayfs penalty on the database (it is a bind mount).
Everything above is machine-facing, and for a long time that was the entire site. Hitting the domain in a browser returned {"detail":"Not Found"}, because I never defined a route for /.
That was fine while the address lived only in my shell history. The moment I decided to write about it publicly, it stopped being fine — the first thing a reader does with a URL is type it into a browser, and Swagger at /docs answers "here are the parameters," not "here is what this service is."
So there is now a landing page at the root. Deliberately unremarkable: one file, one string of HTML, no template engine, no static mount, no build step. It pulls nothing from a CDN — no fonts, no scripts, no images — so it renders identically offline, and the favicon is an inline SVG data URI, which also makes the /favicon.ico request disappear.
Two decisions in it are worth naming, because both are about being honest rather than being polished.
The attribution needed somewhere a human reads. Until this page existed, the ODbL notice lived in JSON responses and inside GPX and KML metadata — correct, and invisible to anyone who was not already parsing the output. The landing page states the license, credits OpenStreetMap contributors, and spells out the distinction that actually matters to a downstream user: displaying the data produces a Produced Work and carries only the attribution requirement, while redistributing the coordinates likely produces a Derivative Database and pulls in share-alike as well. That is the sort of thing people get wrong by never being told, so it is one paragraph rather than a link. A test asserts the notice is present, so it cannot quietly fall out in a refactor.
The page states what the data is not. It says the coordinates are bounding-box centers rather than surveyed pin positions, and that coverage depends entirely on volunteer mapping quality, which is why every course ships a tier, a confidence value, and a warnings array instead of interpolated guesses. A landing page is exactly where a reader forms expectations, so it is the cheapest possible place to prevent a wrong one.
One small design point I would repeat: the page renders even when the database is missing. It reports the indexed course count when it can and silently drops that one sentence when it cannot, rather than failing with a 503. Database health is what /health is for. If the data layer is down, the introduction and the license notice are precisely the parts that should still be standing.
Shipping the landing page meant redeploying, and two things about that second deploy only became visible then.
cloudflared does not need restarting when the container is replaced. docker compose up -d deletes and recreates the socket file, so its inode changes underneath the tunnel — I expected a few 502s while the connection pool drained. There were none. cloudflared resolves the path when a request arrives, so it simply reconnects. Only a config change warrants a restart.
Cloudflare injects a script into HTML responses. The landing page is 5,982 bytes locally and 6,920 bytes through the edge. The extra 938 bytes are a bot-detection beacon that creates a hidden 1×1 iframe and loads /cdn-cgi/challenge-platform/scripts/jsd/main.js. It is not in my repository, so grepping for the cause would waste an afternoon. It applies to HTML only — the JSON, GPX and KML responses are untouched — but it does mean a page I deliberately built to load nothing third-party now loads something third-party, which is worth knowing before you claim otherwise on a privacy page.
Every consumer wants a slightly different shape of this data. That is an obvious temptation to point an LLM at the JSON and let users describe the format they want. I decided to measure it instead of guessing.
The task: convert 18 holes — 72 coordinate values — into a specific CSV layout, checked against a file produced by deterministic code.
| Setup | Result |
|---|---|
| 120B model, raw completion API, no tools | 5,771 tokens consumed, 0 rows produced |
| ~8B model, agent harness with code execution | 72/72 exact |
The larger model never emitted a single CSV row. It narrated its rounding logic — "the seventh digit is 8, so round up" — until it hit the token limit. Doubling max_tokens did not help. Neither did instructing it not to explain.
If that failure mode sounds familiar, it is the one I ran into writing about LLMs and Rubik's cubes on this blog a few weeks ago: a 120B model compulsively simulating everything in its head, burning the entire token budget, and never emitting an action. Different task, identical shape.
Buried in that narration was a genuine arithmetic error: it rounded 35.5764995 to 35.576500, where the correct answer is 35.576499, because the stored float is actually 35.57649949999.... That is about 5.5cm — harmless here, and precisely the kind of wrong answer that looks right.
The small model in an agent harness wrote four lines of Python and ran them. It was exact because Python is exact.
The deciding variable was tool access, not parameter count. That reframes the whole feature: the risk moves from "the model hallucinates coordinates" to "the model runs code I need to sandbox," and the second problem is far more tractable — timeouts, memory caps, no network, and you can return the generated code alongside the output so the user can audit it.
I shelved the feature anyway. Four fixed exporters plus the CLI flag cover the realistic formats, and I confirmed that an ordinary consumer with no database access can produce an arbitrary format from the public API alone — I did exactly that with fifteen lines of browser JavaScript and got 95 out of 95 fields correct. The LLM layer would be solving a problem the API had already solved.
Environments that differ structurally find different bugs. The rate limiter was correct on TCP and broken on a Unix socket. No amount of additional local testing on the same transport would have surfaced it.
Let licensing constrain the design early. "Does the attribution fit inside this format?" turned out to be a clean, mechanical rule for deciding what to expose publicly and what stays a local tool.
Say what the data does not support. 28.5% of these courses are poorly mapped. Publishing that in the response was more useful than any amount of interpolation would have been.
Measure before building the impressive thing. Two hours of testing replaced a whole speculative LLM subsystem with one table.
A public URL needs a front door. The API was complete and correct while its root returned a 404, and I did not notice until I planned to hand the address to strangers. The landing page took an hour and is the only part of the project most visitors will ever see.
The extraction pipeline, the API, the Docker and tunnel configuration, and the full deployment guide all live in the project repository. The service itself answers at golfclubxy.willconia.com, with interactive documentation at /docs.
Disclaimer: This blog post was created with assistance from Claude, an AI developed by Anthropic, under my direct supervision and guidance to ensure accuracy and alignment with my vision for the content.