Skip to main content

​
Your Command Center: UserSessions

Remember Alex from our authentication story? Now that they’re connected, the cloud creates a UserSession - their personal command center. Think of it as a smart assistant that manages everything happening with their glasses.

​
What is a UserSession?

A UserSession is the heart of a user’s experience in MentraOS Cloud. It’s a long-lived object that:
  • Maintains the WebSocket connection to the glasses
  • Manages all the specialized services (managers)
  • Tracks which apps are running
  • Handles all message routing
  • Survives temporary disconnections
// Simplified view of UserSession
class UserSession {
  userId: string;                    // [email protected]
  websocket: WebSocket;              // Connection to glasses
  appWebsockets: Map<string, WebSocket>; // Connections to apps
  startTime: Date;                   // When session started
  
  // The managers (our departments)
  appManager: AppManager;
  audioManager: AudioManager;
  displayManager: DisplayManager;
  // ... and more
}

​
Meet the Managers

Think of managers as specialized departments in Alex’s personal cloud office. Each one handles a specific type of functionality:

​
🎯 AppManager - The App Orchestrator

Manages the lifecycle of third-party apps: 
// What it tracks for each app
{
  packageName: "com.translate.app",
  state: "RUNNING" | "GRACE_PERIOD" | "RESURRECTING" | "STOPPING",
  websocket: WebSocket,
  lastHeartbeat: Date
}
Key Responsibilities:
  • Starting and stopping apps
  • Monitoring app health with heartbeats
  • Handling app reconnections
  • Managing the 20-second grace period when apps disconnect
Real Example: When Alex says “Start translator”, AppManager:
  1. Finds the translator app in installed apps
  2. Notifies the app via webhook to start
  3. Waits for app to connect via WebSocket
  4. Monitors the app’s health

​
🎤 AudioManager - The Sound Department

Handles all things audio: 
// Managing audio streams
audioManager.startStreaming();  // Start receiving audio from glasses
audioManager.stopStreaming();   // Stop audio stream
audioManager.playAudio(audioData, format); // Play audio on glasses
Key Responsibilities:
  • Receiving audio streams from glasses microphone
  • Buffering audio data for processing
  • Sending audio to glasses speakers
  • Managing audio format conversions

​
📺 DisplayManager - The Visual Controller

Controls what appears on the glasses display: 
// Display update request
displayManager.updateDisplay({
  type: "cards",
  cards: [{
    title: "Weather",
    body: "72°F and sunny"
  }]
});
Key Responsibilities:
  • Rendering layouts from apps
  • Enforcing the 200-300ms throttle between updates
  • Managing display queues
  • Preventing display spam
Important: The glasses have a simple display that can only show text. No images, no colors (except green), just text and simple layouts.

​
🎙️ TranscriptionManager - The Speech-to-Text Expert

Converts speech to text in real-time: 
// Start transcribing
transcriptionManager.startTranscription("en-US");

// Receive transcription events
transcriptionManager.on('transcription', (text) => {
  console.log("User said:", text);
});
Key Responsibilities:
  • Managing transcription providers (Deepgram, Assembly, etc.)
  • Streaming audio to transcription services
  • Handling multiple language streams
  • Distributing transcripts to subscribed apps

​
📸 PhotoManager - The Camera Controller

Handles photo capture: 
// App requests a photo
photoManager.requestPhoto();

// Photo arrives from glasses
photoManager.on('photoReceived', (photoData) => {
  // Process and distribute to requesting app
});

​
🌐 MicrophoneManager - The Audio Gate

Controls the glasses microphone: 
// Turn microphone on/off
microphoneManager.setMicrophoneState(true);  // On
microphoneManager.setMicrophoneState(false); // Off

​
📊 DashboardManager - The Always-On Display

Manages the dashboard mode (always-visible information): 
// Update dashboard content
dashboardManager.updateContent({
  text: "3 new messages",
  priority: "high"
});

​
How Managers Work Together

Let’s see how managers collaborate when Alex says “Take a photo”:
  1. AudioManager receives the audio stream
  2. TranscriptionManager converts it to text: “Take a photo”
  3. AppManager routes the transcript to subscribed apps
  4. App recognizes the command and tells PhotoManager
  5. PhotoManager sends photo request to glasses
  6. Glasses capture photo and send it back
  7. PhotoManager receives photo and gives it to the app
  8. App processes photo and uses DisplayManager to show “Photo saved!”

​
Session Lifecycle

​
Birth: Session Creation

When Alex connects: 
const session = new UserSession(userId, websocket);
// All managers are initialized
session.appManager = new AppManager(session);
session.audioManager = new AudioManager(session);
// ... etc

​
Life: Active Session

During normal operation:
  • Heartbeat runs every 10 seconds
  • Messages flow between glasses and apps
  • Managers handle their specific duties
  • Everything is logged for debugging

​
Near-Death: Disconnection

When connection drops:
  • 30-second grace period begins
  • Apps enter “resurrecting” state
  • Session waits for reconnection
  • No data is lost

​
Resurrection or Death

If Alex reconnects within 30 seconds:
  • WebSocket is updated
  • Apps are notified
  • Everything continues normally
If Alex doesn’t reconnect:
  • Session is cleaned up
  • All managers are destroyed
  • Apps are notified of session end
  • Memory is freed

​
Session Storage

All active sessions live in SessionStorage: 
class SessionStorage {
  private static sessions = new Map<string, UserSession>();
  
  static getSession(userId: string): UserSession | undefined {
    return this.sessions.get(userId);
  }
  
  static addSession(session: UserSession) {
    this.sessions.set(session.userId, session);
  }
}
This allows any part of the system to find Alex’s session when needed.

​
Best Practices

  1. Always check session exists before using it
  2. Use the appropriate manager for each task
  3. Let managers handle their domain - don’t bypass them
  4. Clean up resources when sessions end
  5. Log important events for debugging

​
What’s Next?

Now that you understand UserSessions and their managers, let’s look at the messages that flow through the system. These messages are how all the components talk to each other!