Muse — a self-aware AI assistant that lives on your phone.

1 Engine

The Kotlin LLMEngine interface exposes a single streaming method. NativeEngine implements it by calling into jni_interface.cpp, which drives llama.cpp.

interface LLMEngine {
    fun generateStream(
        prompt: String,
        opts: GenOptions,
        onStatus: (Status) -> Unit = {}
    ): Flow<String>
}

Model families & prompt formats

• Gemma Instruct → <start_of_turn>system / user / model
• Qwen (ChatML) → <|im_start|>system / user / assistant
• LLaMA / Tiny → ### SYSTEM / ### CONVERSATION / ### RESPONSE
• Universal fallback uses the ### SYSTEM format
• Detection uses word-boundary regex via isGemmaInstruct()

Sampling options

GenOptions carries maxNewTokens, temperature, topP, topK, repeatPenalty, parseSpecial and addSpecial. The right preset is chosen via chooseGenOptionsForModel(modelName) which reads the user's saved preset from PresetStore.

Load lifecycle

ModelManager (Hilt-scoped singleton) owns loading and exposes loadState: StateFlow<ModelLoadState> with states Idle · Loading · Ready(modelName) · Error.

• ModelDownloadService uses PARTIAL_WAKE_LOCK — never sleeps mid-download
• HTTP Range header support — resumes from .part files
• One notification per model (unique ID), shows speed, bytes, %
• Token expiry (401/403) → deletes .part and prompts re-download
• External GGUF import through Storage Access Framework
• After two failures, the app shows a copyable URL
• Checks available RAM and warns before starting oversized downloads
• Pause + Stop buttons in notification and UI
• DownloadProgressBus — a SharedFlow bridging Service → ViewModel

1 ChatViewModel state

• chatMessages: StateFlow<List<ChatMessage>>
• currentStreamingReply: StateFlow<String?> — live token stream
• isGenerating: StateFlow<Boolean>
• promptEval: StateFlow<Pair<Int,Int>?> — shown in top bar
• chatHistory: StateFlow<List<ChatSession>>
• modelLoadState — delegated from ModelManager

2 sendMessage() pipeline

// 1.  Persist user msg to Room + state
// 2.  Route to LAN if receiver mode
// 3.  Run memory extraction
// 4.  Auto-name session (first 40 chars)
// 5.  Assemble context + notification block
// 6.  buildPromptForModel(...)
// 7.  chooseGenOptionsForModel(...)
// 8.  InferenceService.start()          // foreground notif
// 9.  engine.generateStream()  .takeWhile { !threeBlanksInARow() }
// 10. stripThinkTags(tok)               // per-token
// 11. ActionParser + ActionExecutor
// 12. Save bot message to Room
// 13. InferenceService.stop()

3 Post-processing

• stripThinkTags(text) — removes <think>…</think>; handles streaming unclosed tags
• chatMlStopIndex(text) — first ChatML / EOS marker location
• stripChatMlArtifacts(text) — truncates at marker & regex-strips stray <|…|>

• processUserMessage(msg) — extracts and saves facts from user input
• processAiReply(reply) — reads memory tags from AI responses
• getMemorySummary() — compact string injected into every prompt's system block
• Stored as UserMemory Room entities with an active/inactive flag
• User can inspect & delete memories in MemoryScreen
• Master toggle: memory_enabled pref (default on)

• ActionParser.parse(reply) — extracts [ACTION:…] tags, returns (cleanReply, List<Action>)
• ActionExecutor.execute(actions, sessionId, ctx) — runs them locally:
  • Create reminder / alarm
  • Set timer
  • Save task
• Results are pushed through _actionEvent: SharedFlow<String> → a toast

1 Protocol

Line-delimited TCP on port 8765.

2 Host device

• LanHostService — foreground service, survives backgrounding
• Notification: "N connected · X messages served · Code: XXXXXX"
• onClientChange callback → saves to PREF_CONNECTED_CLIENTS
• ViewModel registers OnSharedPreferenceChangeListener for zero-latency UI

3 Receiver device

• connectReceiver() — auth test only, disconnects after OK
• Connection-per-message on inference (no long-lived socket)
• sendMessageViaLan() mirrors sendMessage() exactly — same memory, context, session naming, post-processing
• Uses buildPromptForModel(hostModelName) — format matches host's model family
• Uses chooseGenOptionsForModel(hostModelName) — same sampling params as local

4 Host discovery

scanForHosts() parallel-connects to all 254 IPs in the subnet with a 300 ms timeout per socket. Found hosts are shown as tappable rows on the Receive tab.

• Polls every 2 seconds
• CPU% from /proc/stat
• Temperature from /sys/class/thermal
• GPU% from /sys/class/kgsl
• Exposed as statsFlow(): Flow<DeviceStats>
• Rendered in ChatScreen's top bar as tertiary 10 sp text — e.g. CPU 34% · 42°C

• Serializes ChatSession, ChatMessage, UserMemory, UserProfile via Gson
• Writes to cacheDir/exports/ (must match file_provider_paths.xml)
• Shares via ACTION_SEND + FileProvider — no WRITE permission needed
• importBackup(uri) reads via SAF ContentResolver and upserts records
• BackupState carries both exportError and importError

• Backed by SharedPreferences (file: app_prefs)
• Top-level screen groups: Appearance · AI Behaviour · Data Management · LAN Hosting
• AdvancedSettingsScreen toggles which rows show up in the main settings list

splash → home (BaseScreen)
  └ newChat · chat/{chatId} · modelManager · memory · tasks
  · lanHost · backupRestore · advancedSettings · about · help
  · achievements

• BaseScreen hosts the bottom nav: Chat list · New Chat · Settings · Profile
• ChatScreenMain is the "Chat list" tab; wraps HomeTopBar + recent sessions

Manifest permissions

Notification channels

• lan_host — LAN hosting service (IMPORTANCE_LOW)
• Model download — one per model
• Inference service — running indicator

FileProvider

Authority: ${applicationId}.fileprovider — paths in res/xml/file_provider_paths.xml.