Posted on April 22, 2026
Category: Technology
Tags: mediapipe, golf, python, pose-estimation, fastapi, opencv, video-processing, computer-vision, ai, machine-learning, ffmpeg, golf-swing-analysis
Views: 260
If you've ever tried to analyze your golf swing from a smartphone video, you know the problem well: you hit record, fidget for ten seconds, address the ball, take your swing, then stand watching the ball fly for another fifteen seconds before finally stopping the recording. The actual swing — the thing you care about — is buried somewhere in the middle of a clip that's mostly dead air.
I built Willconia Golf Swing Trimmer to solve exactly that. Upload a raw golf video, and it automatically detects and extracts your first swing, frame-accurate, with no manual editing. This post walks through how the detection logic actually works.
The obvious first instinct is motion detection — compare pixel differences between frames and flag the active window. I considered it. The problem is that motion detection can't distinguish between meaningful motion (the swing) and irrelevant motion (the golfer shifting weight during setup, wind moving the grass, camera shake). It generates too many false positives.
What I actually needed was body semantics — specifically, the trajectory of the golfer's wrists relative to the rest of their body over time. Google's MediaPipe Pose solution gives me 33 body landmarks per frame in normalized screen coordinates, including both wrists and both elbows, and it runs in near real time. That's exactly the signal I needed.
The core insight is simple: a golf swing produces a very distinctive wrist movement pattern. The wrists rise above the elbows during the backswing, drop sharply through impact, then rise again into the finish. If I can reliably track that pattern, I can locate swing boundaries with precision.
Initialization is straightforward. I use model_complexity=2 for the highest accuracy and keep detection and tracking confidence at 0.65.
import cv2
import mediapipe as mp
import numpy as np
mp_pose = mp.solutions.pose
pose = mp_pose.Pose(
static_image_mode=False,
model_complexity=2,
enable_segmentation=False,
min_detection_confidence=0.65,
min_tracking_confidence=0.65
)
static_image_mode=False is important — it tells MediaPipe to use temporal tracking across frames rather than running full detection from scratch on every frame, which is significantly faster for video.
The main function detect_swing_start_end() does four things in sequence:
def detect_swing_start_end(video_path):
cap = cv2.VideoCapture(video_path)
fps = int(cap.get(cv2.CAP_PROP_FPS))
wrist_avg_ys = []
elbow_avg_ys = []
visible_start_frame = None
frame_idx = 0
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
rgb = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
results = pose.process(rgb)
if results.pose_landmarks:
if visible_start_frame is None and is_golfer_fully_visible(
results.pose_landmarks, min_visible=30
):
visible_start_frame = frame_idx
lm = results.pose_landmarks.landmark
avg_wrist_y = (lm[mp_pose.PoseLandmark.LEFT_WRIST].y +
lm[mp_pose.PoseLandmark.RIGHT_WRIST].y) / 2.0
avg_elbow_y = (lm[mp_pose.PoseLandmark.LEFT_ELBOW].y +
lm[mp_pose.PoseLandmark.RIGHT_ELBOW].y) / 2.0
wrist_avg_ys.append(avg_wrist_y)
elbow_avg_ys.append(avg_elbow_y)
else:
wrist_avg_ys.append(np.nan)
elbow_avg_ys.append(np.nan)
frame_idx += 1
cap.release()
A few things worth noting:
is_golfer_fully_visible() requires at least 30 landmarks with visibility > 0.65 before starting detection. This skips frames where the golfer is still walking into position or only partially visible.def is_golfer_fully_visible(landmarks, min_visible=30):
if not landmarks:
return False
visible_count = sum(
1 for lm in landmarks.landmark if lm.visibility > 0.65
)
return visible_count >= min_visible
Raw wrist Y values are noisy frame to frame. A window-5 moving average smooths out per-frame jitter while preserving the overall shape of the swing arc.
def moving_average(data, window=5):
if len(data) < window:
return data[:]
ma = []
for i in range(len(data)):
if i < window - 1:
ma.append(data[i])
else:
window_values = [v for v in data[i-window+1:i+1] if not np.isnan(v)]
ma.append(np.mean(window_values) if window_values else np.nan)
return ma
Note that nan values (frames where pose wasn't detected) are excluded from the window rather than treated as zero. This prevents the smoothed signal from artificially dipping around dropout frames.
With the smoothed signal ready, the algorithm runs three sequential passes. Each uses the same core pattern: count consecutive frames where the wrist is moving in the required direction (consecutive_up), then fire when the count reaches int(fps / 6.0) — about 5 frames at 30 fps. This threshold ensures we're detecting a sustained movement, not a single-frame noise spike.
BackswingTop is where the wrists reach their highest point (lowest Y) before starting to drop through impact. Three conditions must be met simultaneously:
fps/6 consecutive frames — prev_delta < 0wrist_ma[i] < elbow_avg_ys[i] (lower Y = higher position)current_delta > 0consecutive_up = 0
for i in range(visible_start_frame + 5, len(wrist_ma) - 2):
if np.isnan(wrist_ma[i]) or np.isnan(wrist_ma[i-1]) or np.isnan(elbow_avg_ys[i]):
continue
prev_delta = wrist_ma[i-1] - wrist_ma[i-2]
current_delta = wrist_ma[i] - wrist_ma[i-1]
if prev_delta < 0:
consecutive_up += 1
else:
consecutive_up = 0
if (wrist_ma[i] < elbow_avg_ys[i]
and consecutive_up >= int(fps / 6.0)
and current_delta > 0):
backswingtop_frame = i - 2
start_frame = max(visible_start_frame, backswingtop_frame - int(fps * 2.0))
break
backswingtop_frame = i - 2 steps back two frames from detection — by the time the reversal is confirmed the peak has already passed, so we back up slightly. The start_frame is set to 2 seconds before BackswingTop, which captures the address position.
Impact is the mirror of BackswingTop. The wrists have been falling (Y increasing, moving down in frame) for fps/6 frames, they have crossed back below elbow height, and they are now beginning to decelerate.
consecutive_up = 0
for i in range(backswingtop_frame + 2, len(wrist_ma) - 2):
if np.isnan(wrist_ma[i]) or np.isnan(wrist_ma[i-1]) or np.isnan(elbow_avg_ys[i]):
continue
prev_delta = wrist_ma[i-1] - wrist_ma[i-2]
current_delta = wrist_ma[i] - wrist_ma[i-1]
if prev_delta > 0:
consecutive_up += 1
else:
consecutive_up = 0
if (wrist_ma[i] > elbow_avg_ys[i]
and consecutive_up >= int(fps / 6.0)
and current_delta < 0):
impactpos_frame = i - 2
break
The wrist-vs-elbow comparison is the key discriminator. At impact the hands are near hip height — well below the elbows — so wrist_ma[i] > elbow_avg_ys[i] (larger Y = lower position, since Y increases downward).
FinishTop uses exactly the same conditions as BackswingTop — wrists rising above elbows again — but is searched starting from backswingtop_frame + 2. The follow-through mirrors the backswing arc: the wrists rise above the elbows again and then begin to settle.
finishtop_frame = len(wrist_ma) - 1 # fallback: end of video
consecutive_up = 0
for i in range(backswingtop_frame + 2, len(wrist_ma) - 2):
if np.isnan(wrist_ma[i]) or np.isnan(wrist_ma[i-1]) or np.isnan(elbow_avg_ys[i]):
continue
prev_delta = wrist_ma[i-1] - wrist_ma[i-2]
current_delta = wrist_ma[i] - wrist_ma[i-1]
if prev_delta < 0:
consecutive_up += 1
else:
consecutive_up = 0
if (wrist_ma[i] < elbow_avg_ys[i]
and consecutive_up >= int(fps / 6.0)
and current_delta > 0):
finishtop_frame = i - 2
break
The fallback value (len(wrist_ma) - 1) means that if the finish position isn't detected — for example, the golfer walks out of frame before completing follow-through — the clip simply extends to the end of the video rather than crashing.
With the three key frames located, trim boundaries are straightforward:
backswingtop_frame - 2 * fps, clamped to visible_start_framefinishtop_frame + 0.1 * fps, clamped to last frameend_frame = min(finishtop_frame + int(fps * 0.1), len(wrist_avg_ys) - 1)
start_sec = start_frame / fps
end_sec = end_frame / fps
The actual video cut is handed to FFmpeg, which re-encodes with H.264 for broad device compatibility:
cmd = [
'ffmpeg', '-y', '-i', input_video_path,
'-ss', f"{start_sec:.3f}",
'-to', f"{end_sec:.3f}",
'-c:v', 'libx264', '-preset', 'medium', '-crf', '23',
'-c:a', 'aac', '-movflags', '+faststart',
output_path
]
subprocess.run(cmd, check=True, capture_output=True, text=True)
-movflags +faststart moves the MP4 metadata to the front of the file, enabling the video to begin playing in a browser before the full download completes.
Beyond the trimmed clip, the app also extracts a still image at each of the three key frames using FFmpeg's select filter:
cmd_backswingtop = [
'ffmpeg', '-y', '-i', input_video_path,
'-vf', f"select='eq(n,{backswingtop_frame})'",
'-vframes', '1',
png_backswingtop_path
]
subprocess.run(cmd_backswingtop, check=True, capture_output=True, text=True)
The same pattern repeats for impactpos_frame and finishtop_frame. These PNGs give users a freeze-frame view of the three most analytically useful moments: the top of the backswing, the moment of impact, and the top of the follow-through.
If you print wrist Y and its moving average frame by frame (which I did extensively during development), the swing pattern is unmistakable. Below is the actual debug output from a real test video (30 fps, front-facing, smartphone recording). I've picked representative frames from each phase to keep it readable:
Frame Time Raw Y MA Y Note
─────────────────────────────────────────────────────────────────
141 4.70s 0.5072 0.5108 ← trim start (2s before backswing)
155 5.17s 0.5089 0.5082 address, wrists near hips
164 5.47s 0.5147 0.5142 wrist Y at local peak (lowest position)
172 5.73s 0.5019 0.5038 backswing building, wrists rising
185 6.17s 0.4912 0.5115 wrists accelerating upward
195 6.50s 0.2267 0.2703 wrists well above elbows
201 6.70s 0.1482 0.1618 ← BackswingTop ← detected (Frame 201)
209 6.97s 0.2749 0.2177 downswing, wrists falling fast
216 7.20s 0.4882 0.4785 ← ImpactPos ← detected (Frame 216)
220 7.33s 0.2023 0.3595 follow-through, wrists rising again
223 7.43s 0.1839 0.1980 ← FinishTop ← detected (Frame 223)
225 7.50s 0.2067 0.1823 trim end (+0.1s buffer)
And the final summary printed at the end of detect_swing_start_end():
Final Result:
Start: 4.70s at Frame 141
BackSwing Top: 6.70s at Frame 201
Impact Pos: 7.20s at Frame 216
Finish Top: 7.43s at Frame 223
End: 7.53s
BackswingTop-to-ImpactPos: 0.50s
ImpactPos-to-FinishTop: 0.23s
BackswingTop-to-FinishTop: 0.73s
Swing Length: 2.83s
A few things stand out from the real data:
0.2023 → 0.3754). This is the moving average earning its keep — the smoothed MA values stay coherent through that region while the raw signal bounces around.Each detected transition is consistent across different golfers and different swings, as long as the video is front-facing with decent lighting.
A few things that the current approach handles poorly:
consecutive_up >= fps/6 threshold requires around 5 frames per phase at 30 fps. Below 24 fps, phases may span only 3–4 frames and the threshold may never be met cleanly.The backend is FastAPI. When a user uploads a video, a background task calls trim_golf_swing() while the frontend polls /status/{sid} every two seconds. On completion it redirects to a result page showing the original and trimmed clips side by side, along with the three key position PNGs and timing analysis (backswing-to-impact, impact-to-finish).
Credits are stored in SQLite keyed to a browser cookie UID. Payment goes through Stripe Checkout, with a webhook that atomically grants credits in a single transaction to prevent any double-credit race condition.
Try it at golf.willconia.com — 2 free trims per day, no signup required.
Disclaimer: This blog post was created with assistance from Claude, an AI developed by Anthropic, under my direct supervision and guidance to ensure accuracy and alignment with my vision for the content.