Local Development Setup - MentraOS Cloud

Overview

This guide helps you set up MentraOS Cloud on your local machine for development. You’ll learn how to configure the environment, expose your local server using ngrok, and connect the mobile app to your development cloud instance.

Prerequisites

Before starting, ensure you have:

Bun installed (install.bun.sh)
Docker and Docker Compose installed
ngrok account and CLI (ngrok.com)
Git for cloning the repository
MentraOS mobile app installed on your phone
A MentraOS account

Step 1: Clone and Setup

Clone the Repository

git clone https://github.com/Mentra-Community/MentraOS.git
cd MentraOS/cloud

Install Dependencies

bun install

This installs all dependencies for the cloud packages (cloud, sdk, utils, agents, apps).

Step 2: Environment Configuration

Create .env File

Create a .env file in the cloud directory.

For Internal Team Members: Ask on Slack for the current .env file with all API keys configured.For External Contributors: You’ll need to set up accounts with all third-party services listed below to run the cloud successfully.

Here’s a complete example with all variables:

# Base Configuration
NODE_ENV=development
CLOUD_VERSION=2.0.0
PORT=80  # IMPORTANT: Must be 80 for Docker. External access is on 8002

# Services URLs
CLOUD_HOST_NAME=cloud
PUBLIC_HOST_NAME=localhost:8002
CLOUD_PUBLIC_HOST_NAME=localhost:8002
CLOUD_LOCAL_HOST_NAME=cloud

# MongoDB (required)
MONGO_URL=mongodb://localhost:27017/mentraos
# For MongoDB Atlas (cloud):
# MONGO_URL=mongodb+srv://username:[email protected]/mentraos?retryWrites=true&w=majority

# Supabase Auth (required for authentication)
SUPABASE_URL=https://your-project.supabase.co
SUPABASE_ANON_KEY=your-anon-key
SUPABASE_SERVICE_KEY=your-service-key
SUPABASE_SERVICE_ROLE_KEY=your-service-role-key

# JWT Secrets (required - generate secure random strings)
SUPABASE_JWT_SECRET=your-supabase-jwt-secret
AUGMENTOS_AUTH_JWT_SECRET=your-auth-jwt-secret
TPA_AUTH_JWT_PRIVATE_KEY=-----BEGIN PRIVATE KEY-----
Your-RSA-private-key-here
-----END PRIVATE KEY-----

# Transcription Services (choose one or both)
# Azure Speech (recommended)
AZURE_SPEECH_REGION=centralus
AZURE_SPEECH_KEY=your-azure-speech-key

# Soniox (alternative)
SONIOX_API_KEY=your-soniox-key

# LLM Configuration (for AI features)
LLM_MODEL=gpt-4o
LLM_PROVIDER=azure  # or 'openai' or 'anthropic'

# Azure OpenAI (if using Azure)
AZURE_OPENAI_API_KEY=your-azure-openai-key
AZURE_OPENAI_API_INSTANCE_NAME=your-instance-name
AZURE_OPENAI_API_DEPLOYMENT_NAME=gpt-4o
AZURE_OPENAI_API_VERSION=2024-08-01-preview

# OpenAI (if using OpenAI directly)
OPENAI_API_KEY=sk-proj-your-openai-key

# Anthropic (if using Claude)
ANTHROPIC_API_KEY=your-anthropic-key

# Email Service (for notifications)
RESEND_API_KEY=re_your-resend-key

# Cloudflare Stream (for video streaming)
CLOUDFLARE_ACCOUNT_ID=your-cloudflare-account-id
CLOUDFLARE_API_TOKEN=your-cloudflare-api-token

# Search API (for web search features)
SERPAPI_API_KEY=your-serpapi-key

# Analytics (optional)
POSTHOG_PROJECT_API_KEY=phc_your-posthog-key
POSTHOG_HOST=https://us.i.posthog.com

# Monitoring (optional)
SENTRY_DSN=https://[email protected]/project-id
BETTERSTACK_SOURCE_TOKEN=your-betterstack-token

# System Dashboard App
SYSTEM_DASHBOARD_PACKAGE_NAME=system.augmentos.dashboard

# Timezone
TZ=America/Los_Angeles

# ElevenLabs TTS (optional)
ELEVENLABS_API_KEY=your-elevenlabs-key
ELEVENLABS_DEFAULT_VOICE_ID=your-default-voice-id

Essential vs Optional Variables

Essential for basic operation:

PORT=80 - Must be 80 for Docker port mapping to work
NODE_ENV, CLOUD_VERSION
MONGO_URL - MongoDB connection string
SUPABASE_* - All Supabase variables for authentication
AUGMENTOS_AUTH_JWT_SECRET - JWT signing secret
TPA_AUTH_JWT_PRIVATE_KEY - RSA key for app authentication

Required for specific features:

Transcription: AZURE_SPEECH_* or SONIOX_API_KEY
AI features: LLM configuration (Azure OpenAI, OpenAI, or Anthropic)
Email: RESEND_API_KEY
Video streaming: CLOUDFLARE_*
Web search: SERPAPI_API_KEY
TTS: ELEVENLABS_*

Generating Secrets

# Generate random JWT secrets
openssl rand -base64 64

# Generate RSA private key for TPA_AUTH_JWT_PRIVATE_KEY
openssl genrsa -out private.pem 2048
cat private.pem  # Copy this to your .env file

For External Contributors: Setting up Services

Internal team members can skip this section and get the configured .env from Slack.

MongoDB Setup

Option 1: Local MongoDB with Docker

# Add to docker-compose.dev.yml
services:
  mongodb:
    image: mongo:7.0
    ports:
      - "27017:27017"
    volumes:
      - mongo_data:/data/db
    environment:
      - MONGO_INITDB_DATABASE=mentraos

volumes:
  mongo_data:

Option 2: MongoDB Atlas (Recommended - Free Tier)

Sign up at mongodb.com/atlas
Create a free M0 cluster
Add 0.0.0.0/0 to the IP whitelist (for development only)
Create a database user with read/write access
Get your connection string and add to MONGO_URL

Supabase Setup (Required)

Create a project at supabase.com
Go to Settings → API
Copy:
- Project URL → SUPABASE_URL
- anon public key → SUPABASE_ANON_KEY
- service_role secret key → SUPABASE_SERVICE_KEY and SUPABASE_SERVICE_ROLE_KEY
Go to Settings → Database → Secrets
Copy the JWT Secret → SUPABASE_JWT_SECRET

Azure Speech Setup (For Transcription)

Create an Azure account with free credits
Create a Speech Service resource
Copy the key and region to AZURE_SPEECH_KEY and AZURE_SPEECH_REGION

LLM Provider Setup (Choose One)

Azure OpenAI:

Request access to Azure OpenAI
Deploy a GPT-4 model
Copy credentials to the Azure OpenAI variables

OpenAI:

Create account at platform.openai.com
Generate API key → OPENAI_API_KEY

Anthropic:

Create account at console.anthropic.com
Generate API key → ANTHROPIC_API_KEY

Other Services (Optional)

Resend (Email): Sign up at resend.com
Cloudflare Stream (Video): Requires paid Cloudflare account
SerpAPI (Search): Sign up at serpapi.com
PostHog (Analytics): Sign up at posthog.com
Sentry (Error tracking): Sign up at sentry.io

Step 3: Start the Cloud with Docker

The cloud runs in Docker with all dependencies:

cd MentraOS/cloud/
# then
bun install
# then:
bun run dev

This:

Builds and starts the cloud service in Docker
Maps internal port 80 (from .env) to external port 8002 (from docker-compose.dev.yml)
Mounts your code for hot reloading
Installs dependencies and builds the SDK

The PORT=80 in your .env file is critical. Docker expects the service to run on port 80 internally, which it then maps to 8002 externally. If you set a different PORT in .env, the service won’t be accessible.

What Docker Does

The docker-compose.dev.yml file:

Creates a container for the cloud service
Mounts your local code into /app
Shares bun dependencies in a Docker volume
Runs: bun install && cd packages/sdk && bun run build && cd ../cloud && bun install && bun run dev

Verify Services

# Check if cloud is running
docker ps
# Should show: dev-cloud-1

# View logs
bun run logs
# Or: docker-compose -f docker-compose.dev.yml -p dev logs -f cloud

# Check cloud logs specifically
bun run logs:cloud

Step 4: Start the Cloud Server

# Start the development server
bun run dev

# The server will start on port 8002
# You'll see: "🚀 Server running on http://localhost:8002"

Verify Cloud is Running

# Test the health endpoint
curl http://localhost:8002/health

# Should return:
# {"status":"ok"}

Step 5: Expose with ngrok

To connect from the mobile app, expose your local server using ngrok:

Install ngrok

# macOS with Homebrew
brew install ngrok

# Or download from ngrok.com and add to PATH

Configure ngrok (First Time)

# Add your ngrok authtoken
ngrok config add-authtoken YOUR_AUTH_TOKEN

Start ngrok Tunnel

# Expose port 8002
ngrok http 8002

You’ll see output like:

Session Status                online
Account                       [email protected]
Version                       3.0.0
Region                        United States (us)
Forwarding                    https://abc123.ngrok.app -> http://localhost:8002

Important: Copy the HTTPS URL (e.g., https://abc123.ngrok.app)

Step 6: Configure Mobile App

Enable Developer Mode

Open the MentraOS mobile app
Navigate to the Account page
Tap on your account section 10 times quickly
You’ll see a toast: “Developer mode activated”
Force close and restart the mobile app

Configure Cloud URL

After restart, go to Account section
You’ll now see Developer Settings option
Tap Developer Settings
Find Cloud URL field
Enter your ngrok URL with :443 suffix:
```
https://abc123.ngrok.app:443
```
Note: The :443 is required even though it’s the default HTTPS port.
Tap Save or Apply
The app will reconnect to your local cloud

Verify Connection

Watch your cloud logs:

# In a new terminal
bun run logs:cloud

# You should see:
# "New WebSocket connection from glasses"
# "User authenticated: [email protected]"
# "Session created for user: [email protected]"

Step 7: Development Workflow

Hot Reloading

The development server supports hot reloading:

Code changes in packages/cloud/src automatically restart
WebSocket connections will reconnect automatically

Useful Commands

# View all logs
bun run logs

# Follow specific logs
bun run logs:cloud -- -f

# Clean and rebuild
bun run dev:rebuild

# Reset everything
bun run dev:clean

Testing with Real Glasses

Connect glasses to mobile app as usual
All communication now goes through your local cloud
Monitor WebSocket messages in real-time:
```
LOG_LEVEL=debug bun run dev
```

Database Access

MongoDB (Local)

# Connect to MongoDB
mongosh mongodb://localhost:27017/mentraos

# Common queries
show collections
db.users.find()
db.apps.find()
db.organizations.find()

MongoDB Atlas

# Use MongoDB Compass or Atlas web UI
# Or connect with mongosh using your connection string

Common Issues & Solutions

Port 8002 Already in Use

# Find and kill process
lsof -i :8002
kill -9 <PID>

# Or change port in .env
PORT=8003

Docker Connection Refused

# Ensure Docker is running
docker ps

# Reset Docker network
bun run dev:setup-network

# Rebuild containers
bun run dev:rebuild

ngrok Session Expired

Free ngrok sessions expire after 2 hours
Restart ngrok and update mobile app URL
Consider ngrok paid plan for stable domain

Mobile App Can’t Connect

Check ngrok is running and URL is correct
Include :443 in the URL
Force close and restart mobile app
Check logs for authentication errors
Verify .env has correct JWT secrets

Step 8: Run Web Portals (Store & Developer Console)

MentraOS includes two web applications for app management:

MentraOS Store

The public app store where users discover and install apps:

# Navigate to store directory
cd cloud/store/web

# Install dependencies
bun install

# Start development server
bun run dev

# Opens at http://localhost:5173

Features:

Browse available apps
View app details and permissions
Install/uninstall apps
Search and filter apps
User authentication

Developer Console

Portal for developers to create and manage their apps:

# Navigate to developer portal
cd cloud/developer-portal

# Install dependencies
bun install

# Start development server
bun run dev

# Opens at http://localhost:5174

Features:

Create new apps
Configure app settings and permissions
Manage API keys
Submit apps for review
View analytics and usage
Organization management
Member invitations

Configure Portals for Local Development

Both portals need to connect to your local cloud: 1. Update API endpoints in code: For Store (cloud/store/web/src/api/index.ts):

const API_URL = import.meta.env.DEV 
  ? 'http://localhost:8002/api'
  : 'https://api.mentra.glass/api';

For Developer Console (cloud/developer-portal/src/services/api.service.ts):

const API_URL = import.meta.env.DEV
  ? 'http://localhost:8002/api'
  : 'https://api.mentra.glass/api';

2. Configure Supabase for authentication: Both portals use the same Supabase project as the cloud. They’ll automatically use the environment variables from your cloud .env file.

Access the Portals

Store: http://localhost:5173
- Sign in with your MentraOS account
- Browse and install apps to your account
Developer Console: http://localhost:5174
- Sign in with your developer account
- Create and manage your apps
- Get API keys for testing

Create Test Data

// Connect to MongoDB
mongosh mongodb://localhost:27017/mentraos

// Create test app
db.apps.insertOne({
  packageName: 'com.test.app',
  name: 'Test App',
  logoURL: 'https://example.com/logo.png',
  appType: 'BACKGROUND',
  developerId: '[email protected]',
  hashedApiKey: 'test-api-key-hash',
  permissions: [
    { type: 'CAMERA', description: 'Test camera access' }
  ],
  isPublic: true,
  appStoreStatus: 'DEVELOPMENT'
})

Step 9: Tips for Productive Development

Use multiple terminals:
- Terminal 1: bun run dev (cloud server)
- Terminal 2: ngrok http 8002 (tunnel)
- Terminal 3: bun run logs:cloud -- -f (logs)

VSCode debugging:

// .vscode/launch.json
{
  "type": "bun",
  "request": "launch",
  "name": "Debug Cloud",
  "program": "${workspaceFolder}/packages/cloud/src/index.ts",
  "env": {
    "NODE_ENV": "development"
  }
}

Git workflow:

# Create feature branch
git checkout -b feature/my-feature

# Test thoroughly with local setup
# Commit and push

Security Reminders

Never commit .env files
Don’t share ngrok URLs with sensitive data
Use strong JWT secrets in production
Rotate secrets regularly

Next Steps

Review Cloud Architecture
Understand Message Types
Learn about UserSession
Join Discord for help

Remember: This setup is for development only. Production deployments require proper security, scaling, and monitoring configurations.

Getting Started

Cloud Overview

Cloud Architecture

SDK Reference

Development

​Overview

​Prerequisites

​Step 1: Clone and Setup

​Clone the Repository

​Install Dependencies

​Step 2: Environment Configuration

​Create .env File

​Essential vs Optional Variables

​Generating Secrets

​For External Contributors: Setting up Services

​MongoDB Setup

​Supabase Setup (Required)

​Azure Speech Setup (For Transcription)

​LLM Provider Setup (Choose One)

​Other Services (Optional)

​Step 3: Start the Cloud with Docker

​What Docker Does

​Verify Services

​Step 4: Start the Cloud Server

​Verify Cloud is Running

​Step 5: Expose with ngrok

​Install ngrok

​Configure ngrok (First Time)

​Start ngrok Tunnel

​Step 6: Configure Mobile App

​Enable Developer Mode

​Configure Cloud URL

​Verify Connection

​Step 7: Development Workflow

​Hot Reloading

​Useful Commands

​Testing with Real Glasses

​Database Access

​Common Issues & Solutions

​Port 8002 Already in Use

​Docker Connection Refused

​ngrok Session Expired

​Mobile App Can’t Connect

​Step 8: Run Web Portals (Store & Developer Console)

​MentraOS Store

​Developer Console

​Configure Portals for Local Development

​Access the Portals

​Create Test Data

​Step 9: Tips for Productive Development

​Security Reminders

​Next Steps

Overview

Prerequisites

Step 1: Clone and Setup

Clone the Repository

Install Dependencies

Step 2: Environment Configuration

Create .env File

Essential vs Optional Variables

Generating Secrets

For External Contributors: Setting up Services

MongoDB Setup

Supabase Setup (Required)

Azure Speech Setup (For Transcription)

LLM Provider Setup (Choose One)

Other Services (Optional)

Step 3: Start the Cloud with Docker

What Docker Does

Verify Services

Step 4: Start the Cloud Server

Verify Cloud is Running

Step 5: Expose with ngrok

Install ngrok

Configure ngrok (First Time)

Start ngrok Tunnel

Step 6: Configure Mobile App

Enable Developer Mode

Configure Cloud URL

Verify Connection

Step 7: Development Workflow

Hot Reloading

Useful Commands

Testing with Real Glasses

Database Access

Common Issues & Solutions

Port 8002 Already in Use

Docker Connection Refused

ngrok Session Expired

Mobile App Can’t Connect

Step 8: Run Web Portals (Store & Developer Console)

MentraOS Store

Developer Console

Configure Portals for Local Development

Access the Portals

Create Test Data

Step 9: Tips for Productive Development

Security Reminders

Next Steps