Replaying a Golf Hole in 3D From a GPS Track: Where Did Each Shot Go?

GPS4Golf, my Android golf GPS app, now plays a recorded round back as a 3D replay. A golfer walks the hole behind a push cart, tees off, plays the approach, putts and holes out, all on real terrain and aerial imagery. Here is a one-minute example on YouTube.

To draw that, the replay needs a position for every stroke. The phone almost never has one. This post is about how the replay fills in those positions, and about the one rule that shaped all the others: use a measurement when there is one, use a clearly staged default when there is not, and never invent a measurement.

What the phone actually knows

After a round the app holds five kinds of data, and they differ a lot in how far they can be trusted.

The scorecard. Strokes and putts per hole, entered by the player. This is the ground truth for how many strokes there were, and nothing else.
Marks. The player taps Mark at the ball before a shot. When a Mark exists it is the most accurate position the app has. But most shots have no Mark, and the first Mark of a hole is sometimes made on the previous green.
Swing events. The motion sensors log fast rotations with a time and a GPS position. My previous post explains why these cannot tell a practice swing from a real one. They also only exist when the phone is on the player, not in the cart.
The GPS track. A point every few seconds, accurate to a few metres. It shows where the player walked and stood. It does not show putts. A putt moves the ball, not the player.
The course data. Tee and pin coordinates for each hole. Some holes have no tee in the data at all.

The scorecard says how many shots there were, but nothing says where most of them were hit. The rest of this post is about closing that gap without lying about it.

Full shots: choose from the swings, by count

For every shot off the green, the replay uses the first source available, in this order:

The player's Mark.
A swing site chosen from the swing events.
A staged chip from the last known position. Its landing spot is an estimate, not a measurement.

The middle step does most of the work, and it is a selection problem. The swing events of one hole are a messy list: practice swings on the tee, a waggle while waiting, the real shots, and a few swings that belong to the hole before. The scorecard says how many full shots to keep, the strokes minus the putts. The job is to pick that many from the list.

The rules, in order:

Group the swings. Standing swings that are close in time and place become one site. The site's position is the last swing of the group, because the real shot comes after the practice swings.
Drop what cannot be a full shot. Sites close to the green are chips or putting practice. Sites well behind the tee are waiting on the previous hole.
Collapse repeats. Consecutive sites at the same spot become one. That is the player waiting on a tee and swinging more than once.
Require progress. A full shot moves the ball toward the pin. If a site is not meaningfully closer to the pin than the one before it, the later site replaces the earlier one, since the later swing is the one that hit the ball.
Trim to the count. If there are still more sites than the scorecard allows, drop from the front. The extra early ones were warm-up swings before the hole really started.
Never pad. If there are fewer sites than shots, keep them all and leave the gap. The missing stroke falls back to the staged chip.

How well it matches the shots I recorded

I checked the rules against one 18-hole round. The answer key has 26 known shot positions: 17 shots I had Marked, plus the tee shot on the 9 holes whose tee is in the course data.

Known shots found: 25 of 26, before and after selection. The rules did not throw away real shots.
Distance from the Marks: median about 4 m, worst 9 m. Both ends of that comparison are phone GPS fixes, taken at different moments: the swing at address, the Mark when I tapped beside the ball. Their errors add up, so this is not the accuracy of a distance the app shows.
Distance from the course tees: median about 8 m, worst 18.8 m. Here the answer key is the weak side. The course data has one point per tee, while a tee box is long and the markers move from day to day, so I rarely teed off at that exact point.
Number of sites: the raw groups gave 60 sites for 47 real full shots, 13 too many. After selection there were 37, 10 too few.
Holes with exactly the right count: 6 of 18 before, 9 of 18 after.
The 4 holes where every full-shot position was known: zero wrong choices.

The move from too many to too few is the important part. A site that is too many is an invented shot in the middle of the fairway, drawn as if it happened. A site that is too few is an honest gap, filled by a staged chip that does not pretend to be a measurement. When the data cannot support both, I would rather be short.

The thresholds behind those rules were set from that same round, so they are probably overfitted. The checking script runs again as more rounds come in.

The tee: a chain of fallbacks

The tee looks like the easiest position to know, but it causes the most trouble. The replay tries these in order:

Where the player stood for the first Mark of the hole, unless that spot is on the previous hole's green.
The tee shot the app detected from the swing sensors, using the same rule the app uses on the course.
The tee coordinate in the course data.
The first swing site of the hole that is off both this hole's green and the previous one.
The first GPS point of the hole.

Both "previous green" conditions come from real mistakes in my test round.

Step 1, a stray Mark. On the 2nd hole, my first Mark started on the 1st green: I tapped it after holing out, before walking to the next tee. Taken at face value, it put the 2nd tee on the 1st green. The replay drops that starting point and uses the detected tee shot instead.
Step 4, a stray swing. On the 17th, the first swing logged for the hole was still on the 16th green. Taken at face value, it put the 17th tee on the wrong green.

When the course data has no tee, step 4 has to stand in for it. On that same 17th hole, the substitute tee measured 142 yards to the pin. The scorecard says 145.

Putts: the GPS never saw them

I do not Mark putts, and the app will never ask for them, because tapping a phone over a putt is the fastest way to miss it. The putt count comes from the scorecard, or two putts if the player left it blank. Every putt position is reconstructed.

The cup stays at the course pin. The real cup position that day is unknown, so it would be tempting to move the cup to wherever the data fits best. I tried that and dropped it. The replay's on-screen distance to the pin has to match what the app showed during the round, and moving the cup breaks that.

The first putt starts where the player first lingered on the green. After the last full shot, the replay looks at GPS points inside the green and finds the first place where several consecutive points stay close together. That is the player reading the line and addressing the ball. The ball sits at the average of those points.

That position has to fit the putt count. GPS is accurate to a few metres, and a green is not very big. If the lingering spot is half a metre from the cup but the scorecard says three putts, the replay does not draw a three-putt from half a metre. Each putt count has a plausible range of first-putt distances. A position outside it is moved along the same line until it fits. If the player never lingered at all, the ball is placed on the line from the last shot to the cup, at a typical distance for that putt count.

In the test round, the first putt came from the GPS on 15 holes, from the typical distance on 3, and was moved into range on 3.

The putts that miss are staged. Each miss leaves a distance that grows with the length of the putt, varied slightly per hole, and passes the cup a little to one side. A miss that rolled straight through the middle of the cup would look like a bug.

Rolling putts on the real slope

The green is not flat in the replay. Each one gets a slope from public 1-metre elevation data (USGS 3DEP), and the ball rolls on it: gravity along the slope, friction set for a typical public-course green speed, and a check at the cup for whether the ball is slow enough to drop.

The slope model is a single tilted plane per green, fitted to the elevation points near the pin. I tried two richer models and dropped both:

A curved (quadratic) surface over a wider area picked up the mounds around the edge of the green. Slopes at the edge came out between 9 and 15 percent, which is steeper than any green I have putted on. On one hole, 21 of 112 test putts that should have dropped could not be holed by physics alone and needed their path bent toward the cup, by up to 46 cm. A weighted version of the same fit still produced 15 percent slopes.
Smoothing the slope and capping it fixed the numbers by changing the green. It made the terrain look better by making it less true.

The plane fit gives slopes between 0.1 and 6.2 percent across the 18 greens of the test round.

Aiming is a search problem. A putt that should miss is aimed again and again until it stops at its planned spot. A putt that should drop searches over both direction and speed. If physics still cannot hole it, the closest path is bent gradually into the cup, so the last putt of every hole always drops. To check how often that fallback is needed, a test mode rolls putts from 7 distances in 16 directions on every green. In the test round, all 2,016 putts dropped by physics alone, with no bending.

The same rules on every hole

None of this is tuned per hole. The only things that differ between holes are the input data (GPS, scorecard, elevation) and a random seed made from the date and the hole number. The randomness touches only staged values: how far a missed putt runs past the cup, how far it drifts sideways, and the wind. Measured values never change. That covers Mark distances, stroke counts and putt counts.

A validation step runs over the whole round after every change:

Errors: strokes that do not add up to the scorecard, more than one missing stroke on a hole, a jump in position that no one could walk, a first putt that is not on the green.
Warnings: a tee-to-pin distance that does not fit the hole's par, a shot that moves away from the pin, a walk that goes backwards.

The test round passes on all 18 holes.

One detail matters only in rounds that start on the 10th hole, which is common at busy courses. Several rules look at "the previous hole", and that has to mean the previous hole played, in scorecard order, not the hole with the number one lower. In a round that starts on the 10th, hole 18 comes right before hole 1.

What I would keep from this

Decide what a gap is allowed to look like before filling it. Every rule here came from the choice between drawing an invented shot and admitting a missing one. Once that was settled, most of the thresholds followed.
Score the reconstruction against the few answers you have. Twenty-six known positions are not much, but they turned "this looks right" into 25 of 26 found, and they showed which errors came from the detection and which came from the answer key.
Prefer too few to too many. An extra shot is a lie the viewer cannot detect. A missing one is a gap the replay can label.
Do not fix terrain to make numbers look better. Smoothing and capping the slope produced nicer numbers by changing the green. The simpler plane fit was the right model.
Keep the on-screen numbers equal to the app's. The cup stayed at the course pin even though moving it would have fit the data better. A replay that disagrees with the scorecard the player saw during the round is not worth watching.

The app is GPS4Golf on Google Play. The earlier posts in this series cover the course data and swing detection.

References

Disclaimer: This blog post was created with assistance from Claude, an AI developed by Anthropic, under my direct supervision and guidance to ensure accuracy and alignment with my vision for the content.