WebView / native messages

Message types emitted by Upload Tracking and when they fire.

Upload Tracking emits structured messages so the host app can react.

Transport depends on your integration.

In a browser iframe, it’s postMessage.

In mobile apps, it’s your WebView bridge.

Some bridges stringify the payload. Handle both forms.

Typical V3 flow (Upload Tracking)

Video (source=video)

1. initialization while the page and video load.

2. posture until the exercise is ready.

3. During playback:

   • counter

   • optional form_score, progression, recommendations

   • optional keypoints / angles

4. End of video:

   • counter with final=true

   • optional keypoints_batch (if enabled)

   • optional export_ready (if export=true)

Image (source=image)

1. initialization while the image loads.

2. Optional keypoints / angles.

3. image_overlay with a base64 overlay image.

Message types

initialization (video + image)

Fields:

• type (string) — "initialization".

• message (string) — status.

• ready (boolean) — true when analysis can run.

• source (string, optional) — "video" or "image".

Example:

{
  "type": "initialization",
  "source": "video",
  "message": "Loading video",
  "ready": false
}

environment (video + image)

Sent after backend init.

Fields:

• type (string) — "environment".

• poseBackend (string) — "webgl" or "wasm".

• webgl (boolean) — whether WebGL is available.

Example:

{
  "type": "environment",
  "poseBackend": "webgl",
  "webgl": true
}

error (video + image)

Fields:

• type (string) — "error".

• code (string) — machine-readable error code.

• message (string) — human-readable description.

• details (object, optional).

Example:

{
  "type": "error",
  "code": "video_load_error",
  "message": "Failed to load video",
  "details": {
    "videoSrc": "https://your-cdn.com/video.mp4"
  }
}

Common codes you should handle:

• missing_token

• invalid_token

• invalid_exercise (video only)

• developer_features_not_allowed

• video_load_error

• image_load_error

• media_decode_error

• cross_origin_video

• cross_origin_image

• webgl_unavailable

• jump_analysis_missing_height (for exercise=jump_analysis)

Video-only messages (source=video)

posture

Same meaning as Tracking placement.

{
  "type": "posture",
  "message": "Face the camera. Keep your full body in frame.",
  "direction": "in-frame",
  "ready": false,
  "requirements": ["full_body_visible"]
}

counter

Fields:

• type (string) — "counter".

• current_count (number) — reps or seconds.

• final (boolean, optional) — true at end-of-video.

{
  "type": "counter",
  "current_count": 12,
  "final": true
}

form_score (optional)

{
  "type": "form_score",
  "score": 76,
  "label": "good"
}

progression (plan-gated)

{
  "type": "progression",
  "value": 63
}

recommendations (plan-gated)

{
  "type": "recommendations",
  "data": ["Keep your back straight.", "Control the descent."]
}

keypoints (plan-gated)

{
  "type": "keypoints",
  "data": [
    { "name": "nose", "x": 320.1, "y": 80.0, "score": 0.98 },
    { "name": "left_hip", "x": 345.4, "y": 265.2, "score": 0.96 }
  ]
}

angles (plan-gated)

{
  "type": "angles",
  "data": {
    "left_side": { "knee_angle": { "from_hip_to_ankle": 162 } },
    "right_side": { "knee_angle": { "from_hip_to_ankle": 167 } }
  }
}

keypoints_batch (plan-gated)

Sent near the end of a video analysis.

It contains time-stamped keypoints for the whole clip.

Fields:

• type (string) — "keypoints_batch".

• fps (number) — sampling rate.

• frames (array<object>) — per-frame keypoints.

Example (truncated):

{
  "type": "keypoints_batch",
  "fps": 30,
  "frames": [
    {
      "t": 0,
      "keypoints": [{ "name": "nose", "x": 321.2, "y": 79.9, "score": 0.99 }]
    },
    {
      "t": 33,
      "keypoints": [{ "name": "nose", "x": 320.9, "y": 80.4, "score": 0.99 }]
    }
  ]
}

export_ready (plan-gated, export=true)

Sent when export finishes.

Fields:

• type (string) — "export_ready".

• url (string) — a downloadable URL.

• mimeType (string, optional) — example: "video/mp4".

Example:

{
  "type": "export_ready",
  "url": "blob:https://app.posetracker.com/1c7c0a0b-0f1a-4cd6-8e8a-31b4f1b6c2b3",
  "mimeType": "video/mp4"
}

Jump messages (video, custom exercises)

These fire only for:

• exercise=jump_analysis

• exercise=air_time_jump

They can be emitted during analysis.

They are identical to the real-time message types.

jump_calibration (jump_analysis only)

{
  "type": "jump_calibration",
  "ready": false,
  "message": "Hold still"
}

jump_started

{
  "type": "jump_started"
}

jump_discarded

{
  "type": "jump_discarded",
  "reason": "unstable_baseline"
}

jump_height

{
  "type": "jump_height",
  "jumpHeightCm": 33.7,
  "final": true
}

jump_summary

Sent near the end of a video analysis.

{
  "type": "jump_summary",
  "totalJumps": 3,
  "avgJumpHeight": 31.2,
  "maxJumpHeight": 35.4,
  "minJumpHeight": 27.8,
  "final": true
}

Image-only messages (source=image)

image_overlay

Sent after processing completes.

Fields:

• type (string) — "image_overlay".

• dataUrl (string) — base64 PNG or JPEG.

{
  "type": "image_overlay",
  "dataUrl": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAA..."
}

keypoints and angles (plan-gated)

For images, these are emitted once.