Skip to main content

gamemaster

OmniSight System

OmniSight is SkyrimNet’s AI-powered image-to-text system that gives the LLM real, continuous visual awareness of the game world. It captures in-game screenshots automatically or on demand, analyzes them with a vision model, and generates detailed descriptions , injected in the NPC surrounding context and dynamic behaviors.

Core Features​

📸 Automatic Screenshot Capture​

  • Captures images during player-initiated dialogue, when entering new areas, or via the background worker.
  • Provides passive, continuous world awareness without any user interaction.

🎯 Hotkey Target Scan​

  • Press a hotkey to capture and describe the NPC or object under the crosshair.
  • Hold the hotkey to capture the player character.
  • Actor captures override the actor’s static bio “appearance” text with real visual data.

🧠 Vision-Enhanced LLM Awareness​

  • AI can “see” NPCs, posture, clothing, objects, structures, clutter, terrain, and architecture.
  • Automatically respects first-person vs third-person camera perspective so the model never misinterprets the player model.

🖼️ Dashboard Integration​

  • Dashboard displays the latest screenshot and the AI-generated description.
  • All captures are stored on a dedicated OmniSight page for browsing.
  • Useful for debugging and understanding AI/NPC reactions.

Configuration Options​

General​

  • Enable or disable OmniSight globally.

Background Worker​

  • Passive continuous capturing.
  • Adjustable:
    • Check interval
    • Max concurrent captures
    • Cooldown between captures
  • Extremely low-cost by default: < $0.0004 per capture.

Capture Conditions​

  • Capture only when player is idle.
  • Require fully loaded 3D models.
  • Optional daylight-only outdoor capturing.

Subjects​

  • Toggle Actor captures (hotkey-driven).
  • Toggle Location captures (automatic).

Vision Prompt System​

OmniSight uses specialized prompt templates for:

  • Actor descriptions (appearance, visible equipment, build, posture)
  • Location descriptions (architecture, objects, terrain, layout)
  • Immediate action scans (real-time environment + NPC activity)

All templates enforce:

  • Strict grounding in what is visible
  • No UI-based naming or unseen detail inference
  • Authoritative NPC-visible list rules
  • First-/third-person perspective logic
  • Objective, lore-appropriate language

What OmniSight Enables​

  • Dynamic AI responses based on real visual context
  • More lifelike NPC interactions and situational awareness
  • Reduced need for data integration
  • A world where the AI actually sees what is happening