Face-tracked AR product overlays for glasses, makeup, hats, and accessories using WebXR and MediaPipe
WebXR Virtual Try-On Platform is the AI prompt framework for fashion and beauty ecommerce teams building the AR experiences that are proven to increase conversion rates by 40–94% (Shopify data). The framework covers the complete technical stack for face-tracked product try-on: MediaPipe FaceMesh for the 468-landmark face mesh that works on any mobile browser, WebXR immersive-ar for native AR on supported devices, GLTF product overlay anchoring to specific face landmarks, the occlusion depth mask shader that makes glasses appear to sit on the nose correctly, WebXR light estimation for environment-matched product shading, and the social sharing pipeline that turns every try-on into a marketing moment.
Full Access Unlocked
All 48 prompts · All 6 modules
"The glasses anchor prompt — using the nose bridge and ear landmarks with the correct pivot point — i..."
VP of Digital Commerce · Global eyewear brand, 800+ SKUs
Need expert implementation?
Hire an OrchestratorConnect with a certified Prompt.Doctor Orchestrator to deploy this framework for you.
No coding required. You will use ChatGPT or Claude as your AI tool. Follow these steps in order — do not skip ahead.
Purchase & download the framework
Click the buy button on this page. After checkout, go to the and hit Download .zip. Unzip it — you'll get a .md file (the full framework) and a .pdf (easy to read reference). Keep both open.
Open your project — new or existing
This dashboard is designed to integrate into any existing project or be built as a standalone app. If you already have a site in Airo (or Cursor, Bolt, etc.), open that project. If you're starting fresh, create a new project. The Orchestrator Prompt handles both cases — it scans what's already there and adds only what's missing.
Paste the Orchestrator Prompt into your builder's chat
Open the on this page. Copy the Orchestrator Prompt and paste it into your AI builder's chat. It will scaffold the full admin system — secure login, email marketing module, booking engine, and CMS — on top of your existing codebase. This takes 2–5 minutes.
Add your API keys as secrets
Critical — Novice UsersIn your builder, go to Settings → Secrets and add the keys your app needs. For this framework: STRIPE_SECRET_KEY (for booking payments — get it from your Stripe dashboard), ANTHROPIC_API_KEY (for AI-assisted content — get it from console.anthropic.com), and DATABASE_URL (your MySQL connection string). No key is needed for the admin login, CMS, or email modules — those run on your existing infrastructure.
Don't have a MySQL server?
You can purchase a shared hosting plan with cPanel and MySQL at host.esgwon.dev. Once your account is set up, follow the step-by-step guide to create your database and connect it to your AI website builder.
How to set up cPanel MySQL & connect to your AI website →Need help with Stripe?
Get your STRIPE_SECRET_KEY and STRIPE_PUBLISHABLE_KEY from your Stripe dashboard. The guide covers test keys, webhook setup, and going live.
Need help with Anthropic?
Get your ANTHROPIC_API_KEY from console.anthropic.com. The guide covers model selection, cost management, and troubleshooting.
Prompting Airo after setup — always name the file
When asking Airo to add tables, columns, or features to the admin dashboard, always include src/server/lib/admin-db.ts in your prompt. Without it, Airo may target the wrong database. Example: "Add a bookings table in the admin database (src/server/lib/admin-db.ts) — do not touch any other database connection in this project."
Run the framework prompts inside your live app
Your app is now running in the builder's preview panel. Open the on this page, copy each prompt one at a time, and paste it into your builder's chat. Replace every [BRACKET] with your real data before sending. Work through the stages in order — each stage output feeds the next.
Test end-to-end, then publish or hand off to your client
Walk through the admin as a real user: log in, create a booking, send a test email campaign, update a CMS image, and run the Safe-to-Publish gate. Once everything passes, click Publish in your builder. Because this is a white-label dashboard, your client accesses it at /admin on their own domain — no Prompt.Doctor branding, no third-party login required.
6 modules · 48 prompts · 6 workflow stages
Face Tracking Foundation
MediaPipe FaceMesh setup with 468 landmarks, real-time pose matrix extraction, face bounding box, and landmark group definitions (eyes, nose bridge, ears, forehead)
9 promptsWebXR AR Session
immersive-ar session with face-tracking feature request, WebXR reference space, frame loop with XRFrame, and graceful fallback to MediaPipe for non-WebXR browsers
8 promptsProduct Overlay Anchoring
Glasses anchor to nose bridge + ear landmarks, hat anchor to forehead + crown, earring anchor to ear lobe landmarks, makeup overlay as face mesh texture layer
10 promptsOcclusion & Lighting
Face mesh depth mask shader (render face geometry to depth buffer only), WebXR light estimation probe, environment-matched DirectionalLight and ambient, and shadow casting on face
9 promptsMulti-Product Layering
Product slot system (glasses, hat, earrings, makeup), simultaneous multi-product rendering, z-order management, and product swap animation
7 promptsSharing & Commerce
Canvas composite capture (video + 3D overlay), social share Web Share API, product card overlay on screenshot, and add-to-cart from try-on state
5 promptsWrite a React component that uses MediaPipe FaceMesh to track a user's face in real time and overlays a 3D glasses model anchored to the nose bridge and ears. Requirements: (1) MediaPipe setup: import @mediapipe/face_mesh, create FaceMesh({ maxNumFaces: 1, refineLandmarks: true, minDetectionConfidence: 0.5, minTrackingConfidence: 0.5 }). Connect to a Camera from @mediapipe/camera_utils pointing at a hidden <video> element. (2) On each results callback: extract landmarks[0] — specifically: nose bridge top (landmark 6), nose bridge bottom (landmark 4), left ear (landmark 234), right ear (landmark 454), left eye outer (landmark 33), right eye outer (landmark 263). Convert normalized landmark coordinates to world-space positions using the canvas dimensions and a depth estimate. (3) R3F overlay: a Canvas positioned absolutely over the video feed (pointer-events: none). In useFrame, receive the latest landmarks via a useRef (updated from the MediaPipe callback). Compute: glasses center = midpoint of left/right eye outer landmarks, glasses width = distance between ear landmarks * 0.9, glasses rotation = atan2 of the eye-to-eye vector for roll, head tilt from nose bridge vertical for pitch. Apply position/rotation/scale to the GLTF glasses model. (4) GLTF glasses: load with useGLTF, center the model at the nose bridge pivot point. (5) Export as GlassesARTryOn with prop glassesModelUrl: string. Full TypeScript. Explain the coordinate system conversion from MediaPipe normalized coords to R3F world space.
Write a Three.js / React Three Fiber occlusion system that makes a 3D glasses model appear to sit correctly on a face — with the nose piece behind the nose and the arms behind the ears. Requirements: (1) Face mesh geometry: create a simplified face mesh (use the MediaPipe FaceMesh 468 landmarks to build a BufferGeometry with the canonical face triangulation — provide the index array for the standard MediaPipe face mesh triangles). Update vertex positions each frame from live landmark data. (2) Occlusion material: MeshBasicMaterial({ colorWrite: false, depthWrite: true }) — this renders the face mesh to the depth buffer only, making it invisible but occluding anything behind it. Set renderOrder: 0. (3) Glasses model: set renderOrder: 1 so it renders after the face mesh. The depth test will correctly hide glasses parts that are behind the face geometry. (4) Render order setup: scene.overrideMaterial is NOT used — instead set renderOrder explicitly on each object. (5) Nose bridge occlusion: the nose area landmarks (1, 2, 3, 4, 5, 6, 168, 197, 195, 5) form the nose bridge — ensure these vertices are included in the occlusion mesh with correct triangulation. (6) Provide the complete MediaPipe face mesh triangle indices array (all 468 landmarks, standard triangulation). Full TypeScript with R3F.
system
18 prompts
workflow
13 prompts
templates
10 prompts
prompts
7 prompts
"The glasses anchor prompt — using the nose bridge and ear landmarks with the correct pivot point — is what makes the difference between a try-on that looks like a sticker and one that looks like you're actually wearing the glasses. The coordinate system explanation is worth the price alone."
VP of Digital Commerce
Global eyewear brand, 800+ SKUs
"The occlusion depth mask shader prompt is the most technically correct implementation of face occlusion I've seen documented anywhere. The renderOrder pattern with colorWrite: false is exactly right — and the full MediaPipe triangle indices array saves hours of research."
AR Engineer
Beauty tech platform, 2M monthly try-ons
All 48 prompts across 6 modules are unlocked for your account.
Lifetime access