📱Expo React Native example

How to use PoseTracker in a react native application ?

Base on the schema on How does it work ? You need to call our URL https://app.posetracker.com/pose_tracker/tracking through a WebView.

Base on this here is a tutorial to use PoseTracker in a Mobile App :

News — PoseTracker (iOS): camera permissions and WebView behavior

This covers:

• Request camera access correctly on iOS with expo-camera.

• Avoid the extra WebView popup on iOS 15+: “Allow app.posetracker.com to access the camera”.

• Stay compliant with App Store guideline 5.1.1.

1) Native camera permission (required)

PoseTracker runs inside a WebView.

iOS blocks camera usage until the native app has camera permission.

We use expo-camera:

• useCameraPermissions() to check / request access.

• NSCameraUsageDescription in app.json → ios.infoPlist.

Example:

{
  "expo": {
    "ios": {
      "infoPlist": {
        "NSCameraUsageDescription": "Camera access is required for real-time tracking."
      }
    }
  }
}

Recommended flow (used below):

• Don’t request camera access on app launch.

• Request it only when the user starts a camera feature (ex: “Live camera”).

• If access is denied, explain why.

• Show “Open Settings” only when the user tries again.

2) WebView extra camera prompt on iOS 15+

Even when native permission is granted, iOS may show a second prompt inside the WebView.

To avoid it on iOS 15+, set mediaCapturePermissionGrantType="grant" on react-native-webview.

Conditions:

• iOS 15+.

• Native camera permission is already granted.

Result:

• No extra “Allow … to access the camera” prompt on each WebView mount.

• You can unmount/remount without re-prompting.

3) App Store guideline 5.1.1

To stay compliant:

• Don’t redirect to Settings before showing the system permission dialog.

• Request permission only when the user starts a camera feature.

• If they deny and try again, you can show an explanation and an optional Settings link.

Start a new Expo project

npx create-expo-app my-app
cd my-app
yarn install

Install dependencies for the webview

npx expo install expo-camera react-native-webview

Start and run the project on your Phone or a Simulator

For this part we recommend to use your phone with Expo Go App as described in Expo Documentation. Because we will need to access a camera with a human to track.

npx expo start

// Then open the app on Expo GO

Integrate PoseTracker

Define your App.js and set your API_KEY :

import React, { useCallback, useState } from 'react';
import { Alert, Button, Dimensions, Linking, Platform, StyleSheet, Text, View } from 'react-native';
import WebView from 'react-native-webview';
import { useCameraPermissions } from 'expo-camera';

const API_KEY = "REPLACE_WITH_YOU_API_KEY";
const POSETRACKER_API = "https://app.posetracker.com/pose_tracker/tracking";
const { width, height } = Dimensions.get('window');

export default function App() {
  const [poseTrackerInfos, setCurrentPoseTrackerInfos] = useState();
  const [repsCounter, setRepsCounter] = useState(0);
  const [isLiveEnabled, setIsLiveEnabled] = useState(false);
  const [permission, requestPermission] = useCameraPermissions();

  const ensureRealtimeCameraPermission = useCallback(async () => {
    if (permission?.granted) return true;

    const response = await requestPermission();
    if (response?.granted) return true;

    Alert.alert(
      "Camera required",
      "Camera access is required for real-time tracking. You can enable it in your device settings if you previously denied access.",
      [
        { text: "Cancel", style: "cancel" },
        {
          text: "Open Settings",
          onPress: () => {
            if (Linking.openSettings) {
              Linking.openSettings().catch(() => {});
            }
          },
        },
      ]
    );

    return false;
  }, [permission?.granted, requestPermission]);

  const startLive = useCallback(async () => {
    const ok = await ensureRealtimeCameraPermission();
    if (!ok) return;
    setIsLiveEnabled(true);
  }, [ensureRealtimeCameraPermission]);

  const exercise = "squat";
  const difficulty = "easy";
  const skeleton = true;
  const isAndroid = Platform.OS === "android"
  const posetracker_url = `${POSETRACKER_API}?token=${API_KEY}&isAndroid=${isAndroid}&exercise=${exercise}&difficulty=${difficulty}&width=${width}&height=${height}&isMobile=${true}`;

  // Bridge JavaScript BETWEEN POSETRACKER & YOUR APP
  const jsBridge = `
    window.addEventListener('message', function(event) {
      window.ReactNativeWebView.postMessage(JSON.stringify(event.data));
    });

    window.webViewCallback = function(data) {
      window.ReactNativeWebView.postMessage(JSON.stringify(data));
    };

    const originalPostMessage = window.postMessage;
    window.postMessage = function(data) {
      window.ReactNativeWebView.postMessage(typeof data === 'string' ? data : JSON.stringify(data));
    };

    true; // Important for a correct injection
  `;

  const handleCounter = (count) => {
    setRepsCounter(count);
  };

  const handleInfos = (infos) => {
    setCurrentPoseTrackerInfos(infos);
    console.log('Received infos:', infos);
  };

  const webViewCallback = (info) => {
    if (info?.type === 'counter') {
      handleCounter(info.current_count);
    } else {
      handleInfos(info);
    }
  };

  const onMessage = (event) => {
    try {
      let parsedData;
      if (typeof event.nativeEvent.data === 'string') {
        parsedData = JSON.parse(event.nativeEvent.data);
      } else {
        parsedData = event.nativeEvent.data;
      }

      console.log('Parsed data:', parsedData);
      webViewCallback(parsedData);
    } catch (error) {
      console.error('Error processing message:', error);
      console.log('Problematic data:', event.nativeEvent.data);
    }
  };

  return (
    <View style={styles.container}>
      {!isLiveEnabled ? (
        <View style={styles.ctaContainer}>
          <Text style={styles.ctaTitle}>Ready to start live tracking?</Text>
          <Button title="Start Live camera" onPress={startLive} />
        </View>
      ) : (
        <WebView
          javaScriptEnabled={true}
          domStorageEnabled={true}
          allowsInlineMediaPlayback
          mediaPlaybackRequiresUserAction={false}
          style={styles.webView}
          source={{ uri: posetracker_url }}
          originWhitelist={['*']}
          injectedJavaScript={jsBridge}
          onMessage={onMessage}
          {...(Platform.OS === "ios" && {
            // iOS 15+: avoid the extra “Allow … to access the camera” prompt.
            mediaCapturePermissionGrantType: "grant",
          })}
          // Activer le debug pour voir les logs WebView
          debuggingEnabled={true}
          // Permettre les communications mixtes HTTP/HTTPS si nécessaire
          mixedContentMode="compatibility"
          // Ajouter un gestionnaire d'erreurs
          onError={(syntheticEvent) => {
            const { nativeEvent } = syntheticEvent;
            console.warn('WebView error:', nativeEvent);
          }}
          // Ajouter un gestionnaire pour les erreurs de chargement
          onLoadingError={(syntheticEvent) => {
            const { nativeEvent } = syntheticEvent;
            console.warn('WebView loading error:', nativeEvent);
          }}
        />
      )}
      <View style={styles.infoContainer}>
        <Text>Status : {!poseTrackerInfos ? "loading AI..." : "AI Running"}</Text>
        <Text>Info type : {!poseTrackerInfos ? "loading AI..." : poseTrackerInfos.type}</Text>
        <Text>Counter: {repsCounter}</Text>
        {poseTrackerInfos?.ready === false ? (
          <>
            <Text>Placement ready: false</Text>
            <Text>Placement info: Move {poseTrackerInfos?.postureDirection}</Text>
          </>
        ) : (
          <>
            <Text>Placement ready: true</Text>
            <Text>Placement info: You can start doing squats 🏋️</Text>
          </>
        )}
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    flexDirection: 'column',
  },
  ctaContainer: {
    flex: 1,
    alignItems: 'center',
    justifyContent: 'center',
    padding: 24,
  },
  ctaTitle: {
    marginBottom: 12,
    fontSize: 16,
  },
  webView: {
    width: '100%',
    height: '100%',
    zIndex: 1,
  },
  infoContainer: {
    position: 'absolute',
    top: 60,
    left: 0,
    right: 0,
    alignItems: 'center',
    zIndex: 2,
    backgroundColor: 'rgba(255, 255, 255, 0.8)',
    padding: 10,
  },
});

How does it works ?

• First we have the WebView :

<WebView
        javaScriptEnabled={true}
        domStorageEnabled={true}
        allowsInlineMediaPlayback={true}
        mediaPlaybackRequiresUserAction={false}
        style={styles.webView}
        source={{ uri: posetracker_url }}
        originWhitelist={['*']}
        injectedJavaScript={jsBridge}
        onMessage={onMessage}
        {...(Platform.OS === "ios" && {
          mediaCapturePermissionGrantType: "grant",
        })}
        // Activer le debug pour voir les logs WebView
        debuggingEnabled={true}
        // Permettre les communications mixtes HTTP/HTTPS si nécessaire
        mixedContentMode="compatibility"
        // Ajouter un gestionnaire d'erreurs
        onError={(syntheticEvent) => {
          const { nativeEvent } = syntheticEvent;
          console.warn('WebView error:', nativeEvent);
        }}
        // Ajouter un gestionnaire pour les erreurs de chargement
        onLoadingError={(syntheticEvent) => {
          const { nativeEvent } = syntheticEvent;
          console.warn('WebView loading error:', nativeEvent);
        }}
      />

That use posetracker_url who's build with PoseTracker params.

🟧 Important point : Data exchange between PoseTracker and your Application 🟧

We use what you are sending throw injectedJavaScript to send back data to our App. So we create a bridge between PoseTracker API frontend and YOUR actual App with :

injectedJavaScript={jsBridge}

// This is a basic js bridge
// We need a bridge to transit data between the ReactNative app and our WebView
// The WebView will use this function define here to send info that we will use later

  const jsBridge = `
    window.addEventListener('message', function(event) {
      window.ReactNativeWebView.postMessage(JSON.stringify(event.data));
    });

    window.webViewCallback = function(data) {
      window.ReactNativeWebView.postMessage(JSON.stringify(data));
    };

    const originalPostMessage = window.postMessage;
    window.postMessage = function(data) {
      window.ReactNativeWebView.postMessage(typeof data === 'string' ? data : JSON.stringify(data));
    };

    true; // Important for a correct injection
  `;

Then we use your bridge to send you some information, and here we handle it with:

 const handleCounter = (count) => {
    setRepsCounter(count);
  };

  const handleInfos = (infos) => {
    setCurrentPoseTrackerInfos(infos);
    console.log('Received infos:', infos);
  };

  const webViewCallback = (info) => {
    if (info?.type === 'counter') {
      handleCounter(info.current_count);
    } else {
      handleInfos(info);
    }
  };

You can find all the informations returned by PoseTracker here.

Result

Video of the tutorial App

You can find source files here

This repo is kept as a reference implementation on GitHub.

We’ll clean and stabilize it over time.

It will remain available as sample code for integration.

GitHub - Movelytics/PoseTracker-Example-ReactNative-Expo: Quickly add Pose Estimation and Detection to your React Native app. Enable powerful features in your app powered by your user body tracking !.GitHub