# Expo React Native example

Base on the schema on [how-does-it-work](https://posetracker.gitbook.io/posetracker-api/use-posetracker-on-real-time-camera-webcam/how-does-it-work "mention") You need to call our URL <https://app.posetracker.com/pose_tracker/tracking> through a **WebView.**

**Base on this here is a tutorial to use PoseTracker in a Mobile App :**

{% hint style="info" %}
**News — PoseTracker (iOS): camera permissions and WebView behavior**

This covers:

* Request camera access correctly on iOS with `expo-camera`.
* Avoid the extra WebView popup on iOS 15+: “Allow app.posetracker.com to access the camera”.
* Stay compliant with App Store guideline **5.1.1**.

#### 1) Native camera permission (required)

PoseTracker runs inside a WebView.

iOS blocks camera usage until the **native app** has camera permission.

We use `expo-camera`:

* `useCameraPermissions()` to check / request access.
* `NSCameraUsageDescription` in `app.json` → `ios.infoPlist`.

Example:

```json
{
  "expo": {
    "ios": {
      "infoPlist": {
        "NSCameraUsageDescription": "Camera access is required for real-time tracking."
      }
    }
  }
}
```

Recommended flow (used below):

* Don’t request camera access on app launch.
* Request it only when the user starts a camera feature (ex: “Live camera”).
* If access is denied, explain why.
* Show “Open Settings” only when the user tries again.

#### 2) WebView extra camera prompt on iOS 15+

Even when native permission is granted, iOS may show a second prompt inside the WebView.

To avoid it on iOS 15+, set `mediaCapturePermissionGrantType="grant"` on `react-native-webview`.

Conditions:

* iOS 15+.
* Native camera permission is already granted.

Result:

* No extra “Allow … to access the camera” prompt on each WebView mount.
* You can unmount/remount without re-prompting.

#### 3) App Store guideline 5.1.1

To stay compliant:

* Don’t redirect to Settings before showing the system permission dialog.
* Request permission only when the user starts a camera feature.
* If they deny and try again, you can show an explanation and an optional Settings link.
  {% endhint %}

#### Start a new Expo project

```
npx create-expo-app my-app
cd my-app
yarn install
```

#### Install dependencies for the webview

<pre><code><strong>npx expo install expo-camera react-native-webview
</strong></code></pre>

#### Start and run the project on your Phone or a Simulator

For this part we recommend to use your phone with Expo Go App as described in [Expo Documentation](https://docs.expo.dev/tutorial/create-your-first-app/#run-the-app-on-mobile-and-web). Because we will need to access a camera **with a human to track**.

```
npx expo start

// Then open the app on Expo GO
```

#### Integrate PoseTracker

Define your App.js and <mark style="color:red;">**set your API\_KEY**</mark> :

```
import React, { useCallback, useState } from 'react';
import { Alert, Button, Dimensions, Linking, Platform, StyleSheet, Text, View } from 'react-native';
import WebView from 'react-native-webview';
import { useCameraPermissions } from 'expo-camera';

const API_KEY = "REPLACE_WITH_YOU_API_KEY";
const POSETRACKER_API = "https://app.posetracker.com/pose_tracker/tracking";
const { width, height } = Dimensions.get('window');

export default function App() {
  const [poseTrackerInfos, setCurrentPoseTrackerInfos] = useState();
  const [repsCounter, setRepsCounter] = useState(0);
  const [isLiveEnabled, setIsLiveEnabled] = useState(false);
  const [permission, requestPermission] = useCameraPermissions();

  const ensureRealtimeCameraPermission = useCallback(async () => {
    if (permission?.granted) return true;

    const response = await requestPermission();
    if (response?.granted) return true;

    Alert.alert(
      "Camera required",
      "Camera access is required for real-time tracking. You can enable it in your device settings if you previously denied access.",
      [
        { text: "Cancel", style: "cancel" },
        {
          text: "Open Settings",
          onPress: () => {
            if (Linking.openSettings) {
              Linking.openSettings().catch(() => {});
            }
          },
        },
      ]
    );

    return false;
  }, [permission?.granted, requestPermission]);

  const startLive = useCallback(async () => {
    const ok = await ensureRealtimeCameraPermission();
    if (!ok) return;
    setIsLiveEnabled(true);
  }, [ensureRealtimeCameraPermission]);

  const exercise = "squat";
  const difficulty = "easy";
  const skeleton = true;
  const isAndroid = Platform.OS === "android"
  const posetracker_url = `${POSETRACKER_API}?token=${API_KEY}&isAndroid=${isAndroid}&exercise=${exercise}&difficulty=${difficulty}&width=${width}&height=${height}&isMobile=${true}`;

  // Bridge JavaScript BETWEEN POSETRACKER & YOUR APP
  const jsBridge = `
    window.addEventListener('message', function(event) {
      window.ReactNativeWebView.postMessage(JSON.stringify(event.data));
    });

    window.webViewCallback = function(data) {
      window.ReactNativeWebView.postMessage(JSON.stringify(data));
    };

    const originalPostMessage = window.postMessage;
    window.postMessage = function(data) {
      window.ReactNativeWebView.postMessage(typeof data === 'string' ? data : JSON.stringify(data));
    };

    true; // Important for a correct injection
  `;

  const handleCounter = (count) => {
    setRepsCounter(count);
  };

  const handleInfos = (infos) => {
    setCurrentPoseTrackerInfos(infos);
    console.log('Received infos:', infos);
  };

  const webViewCallback = (info) => {
    if (info?.type === 'counter') {
      handleCounter(info.current_count);
    } else {
      handleInfos(info);
    }
  };

  const onMessage = (event) => {
    try {
      let parsedData;
      if (typeof event.nativeEvent.data === 'string') {
        parsedData = JSON.parse(event.nativeEvent.data);
      } else {
        parsedData = event.nativeEvent.data;
      }

      console.log('Parsed data:', parsedData);
      webViewCallback(parsedData);
    } catch (error) {
      console.error('Error processing message:', error);
      console.log('Problematic data:', event.nativeEvent.data);
    }
  };

  return (
    <View style={styles.container}>
      {!isLiveEnabled ? (
        <View style={styles.ctaContainer}>
          <Text style={styles.ctaTitle}>Ready to start live tracking?</Text>
          <Button title="Start Live camera" onPress={startLive} />
        </View>
      ) : (
        <WebView
          javaScriptEnabled={true}
          domStorageEnabled={true}
          allowsInlineMediaPlayback
          mediaPlaybackRequiresUserAction={false}
          style={styles.webView}
          source={{ uri: posetracker_url }}
          originWhitelist={['*']}
          injectedJavaScript={jsBridge}
          onMessage={onMessage}
          {...(Platform.OS === "ios" && {
            // iOS 15+: avoid the extra “Allow … to access the camera” prompt.
            mediaCapturePermissionGrantType: "grant",
          })}
          // Activer le debug pour voir les logs WebView
          debuggingEnabled={true}
          // Permettre les communications mixtes HTTP/HTTPS si nécessaire
          mixedContentMode="compatibility"
          // Ajouter un gestionnaire d'erreurs
          onError={(syntheticEvent) => {
            const { nativeEvent } = syntheticEvent;
            console.warn('WebView error:', nativeEvent);
          }}
          // Ajouter un gestionnaire pour les erreurs de chargement
          onLoadingError={(syntheticEvent) => {
            const { nativeEvent } = syntheticEvent;
            console.warn('WebView loading error:', nativeEvent);
          }}
        />
      )}
      <View style={styles.infoContainer}>
        <Text>Status : {!poseTrackerInfos ? "loading AI..." : "AI Running"}</Text>
        <Text>Info type : {!poseTrackerInfos ? "loading AI..." : poseTrackerInfos.type}</Text>
        <Text>Counter: {repsCounter}</Text>
        {poseTrackerInfos?.ready === false ? (
          <>
            <Text>Placement ready: false</Text>
            <Text>Placement info: Move {poseTrackerInfos?.postureDirection}</Text>
          </>
        ) : (
          <>
            <Text>Placement ready: true</Text>
            <Text>Placement info: You can start doing squats 🏋️</Text>
          </>
        )}
      </View>
    </View>
  );
}

const styles = StyleSheet.create({
  container: {
    flex: 1,
    flexDirection: 'column',
  },
  ctaContainer: {
    flex: 1,
    alignItems: 'center',
    justifyContent: 'center',
    padding: 24,
  },
  ctaTitle: {
    marginBottom: 12,
    fontSize: 16,
  },
  webView: {
    width: '100%',
    height: '100%',
    zIndex: 1,
  },
  infoContainer: {
    position: 'absolute',
    top: 60,
    left: 0,
    right: 0,
    alignItems: 'center',
    zIndex: 2,
    backgroundColor: 'rgba(255, 255, 255, 0.8)',
    padding: 10,
  },
});
```

#### How does it works ?

* First we have the WebView :

```
<WebView
        javaScriptEnabled={true}
        domStorageEnabled={true}
        allowsInlineMediaPlayback={true}
        mediaPlaybackRequiresUserAction={false}
        style={styles.webView}
        source={{ uri: posetracker_url }}
        originWhitelist={['*']}
        injectedJavaScript={jsBridge}
        onMessage={onMessage}
        {...(Platform.OS === "ios" && {
          mediaCapturePermissionGrantType: "grant",
        })}
        // Activer le debug pour voir les logs WebView
        debuggingEnabled={true}
        // Permettre les communications mixtes HTTP/HTTPS si nécessaire
        mixedContentMode="compatibility"
        // Ajouter un gestionnaire d'erreurs
        onError={(syntheticEvent) => {
          const { nativeEvent } = syntheticEvent;
          console.warn('WebView error:', nativeEvent);
        }}
        // Ajouter un gestionnaire pour les erreurs de chargement
        onLoadingError={(syntheticEvent) => {
          const { nativeEvent } = syntheticEvent;
          console.warn('WebView loading error:', nativeEvent);
        }}
      />
```

That use posetracker\_url who's build with PoseTracker params.

#### 🟧 Important point : D**ata exchange between PoseTracker and your Application 🟧**

We use what you are sending throw `injectedJavaScript` to send back data to our App. So we create a bridge between PoseTracker API frontend and YOUR actual App with :

```
injectedJavaScript={jsBridge}

// This is a basic js bridge
// We need a bridge to transit data between the ReactNative app and our WebView
// The WebView will use this function define here to send info that we will use later

  const jsBridge = `
    window.addEventListener('message', function(event) {
      window.ReactNativeWebView.postMessage(JSON.stringify(event.data));
    });

    window.webViewCallback = function(data) {
      window.ReactNativeWebView.postMessage(JSON.stringify(data));
    };

    const originalPostMessage = window.postMessage;
    window.postMessage = function(data) {
      window.ReactNativeWebView.postMessage(typeof data === 'string' ? data : JSON.stringify(data));
    };

    true; // Important for a correct injection
  `;
```

#### Then we use your bridge to send you some information, and here we handle it with:

```
 const handleCounter = (count) => {
    setRepsCounter(count);
  };

  const handleInfos = (infos) => {
    setCurrentPoseTrackerInfos(infos);
    console.log('Received infos:', infos);
  };

  const webViewCallback = (info) => {
    if (info?.type === 'counter') {
      handleCounter(info.current_count);
    } else {
      handleInfos(info);
    }
  };
```

You can find all the informations returned by [PoseTracker here.](https://posetracker.gitbook.io/posetracker-api/use-posetracker-on-real-time-camera-webcam/tracking-endpoint-message-to-handle)

#### Result

{% embed url="<https://youtube.com/shorts/4JOYh_JUlv0?feature=share>" %}
Video of the tutorial App
{% endembed %}

#### You can find source files here

This repo is kept as a reference implementation on GitHub.

We’ll clean and stabilize it over time.

It will remain available as sample code for integration.

{% embed url="<https://github.com/Movelytics/ReactNativeWebcamDemo>" %}