SilverGait Technical Reference

Agent Pipeline Architecture

Two-graph system for elderly frailty management. Assessment is fully deterministic. Chat orchestrates LLM sub-agents via Gemini function calling.

01

Assessment Graph

0 LLM calls · Fully deterministic · assessment_graph.py

Scores raw health inputs, classifies frailty tier, and conditionally updates care plans and alerts. Every node is deterministic—no LLM variance in critical clinical classification.

Triggers

Onboarding Video Assessment Profile Update Biweekly Recheck
triggers score_node Katz ADL, CFS, SPPB score_katz() → 0-6 score_cfs() → 1-9 score_sppb() → 0-12 classify_node classify_frailty() → tier robust | pre_frail frail | severely_frail tier_change _router changed update_plans_node route_care() → build CarePlans Supersedes old CarePlans in DB notify_node Alert if tier declined / improved No alert on first evaluation unchanged persist_node FrailtyEvaluation + AgentRun SQLite write
Deterministic node
Decision router
Trigger
Changed path
Unchanged (skip)
02

Chat Graph

1–5 LLM calls · Gemini function calling · chat_graph.py

Assembles full user context from the database, invokes Gemini 2.5 Flash with function calling, optionally dispatches to management sub-agents, then applies safety pattern matching before persisting.

user message context_assembly build_user_context() → prompt DB 10+ queries agent_node Gemini 2.5 Flash 1st call → [tool?] → 2nd call streams SSE safety_gate falls, emergency, distress fell / chest pain / want to die jatuh / 跌倒 / 뤵먴ுத → Alert + safety info appended persist save msgs function_call SUB-AGENTS (invoked via Gemini function calling) Exercise Agent Flash Lite · 1 LLM 4-week program Sleep Agent Flash Lite · 1 LLM CBT-I + sleep hygiene Education Agent Flash Lite · 1 LLM Frailty / nutrition Monitoring Agent Flash Lite · 1 LLM Trend analysis get_progress_summary SPPB/Katz trends · 0 LLM alert_caregiver Caregiver notification · 0 LLM function_response 2nd Gemini call → synthesize tool results into response 1–5 LLM total Sub-agents receive UserContext from context_assembly_node. No direct DB access.
Deterministic (0 LLM)
LLM node (Gemini)
function_call
function_response
03

Vision & Voice Pipelines

Gemini Vision · MERaLiON STT · ElevenLabs TTS · fallback chains

Three external API pipelines handle video analysis, speech recognition (with Singlish awareness), and text-to-speech with voice cloning. Each has a cascading fallback to Gemini.

VISION PIPELINE Phone Camera WebM / MP4 video Gemini File API upload & process video analyze Gemini 2.5 Flash Lite Video → SPPB sub-scores (0-4 each) gait + balance + chair stand analysis scores Assessment Graph 0 LLM · score → classify → persist SPEECH-TO-TEXT PIPELINE Microphone WAV / WebM audio MERaLiON AudioLLM cr8lab API · Singlish-aware · code-switching EN / ZH / MS / TA · tuned for elderly speech transcription Text Output + language ID Gemini STT fallback if MERaLiON unavailable fallback TEXT-TO-SPEECH PIPELINE Agent Reply text to speak ElevenLabs Streaming MP3 · expressive · low latency Voice cloning from caregiver audio sample MP3 stream Speaker audio playback Gemini TTS fallback WAV · if ElevenLabs unavailable fallback
User input / output
Gemini API
Third-party API
Internal component
Fallback path
Google AI

Gemini

2.5 Flash · 2.5 Flash Lite

  • Vision — Video SPPB scoring (Flash Lite)
  • Chat Agent — Orchestrator + function calling (Flash)
  • Sub-Agents — 4 management agents (Flash Lite)
  • STT fallback — If MERaLiON unavailable
  • TTS fallback — WAV output if ElevenLabs down
cr8lab

MERaLiON AudioLLM

Singlish-aware speech model

  • Primary STT — Transcription with Singlish accents
  • Code-switching — EN/ZH/MS/TA mixed speech
  • Translation — Audio → target language text
  • Elderly-tuned — Low temp, conservative decoding
ElevenLabs

ElevenLabs

Neural text-to-speech

  • Primary TTS — Streaming MP3 audio
  • Voice cloning — Clone caregiver's voice
  • Expressive — Tunable stability & style
  • Low latency — Real-time streaming playback
04

Data Flow

Assessment writes · Chat reads & writes · SQLite (append-only snapshots)

The Assessment Graph populates the database with evaluations and care plans. The Chat Graph reads this context to personalize every conversation, and writes back chat history and safety alerts.

Assessment Graph 0 LLM · deterministic score → classify → plans → persist Chat Graph 1-5 LLM · Gemini function calling context → agent → safety → persist SQLite silvergait.db WRITES READS WRITES Assessment Graph writes: health_snapshots frailty_evaluations care_plans, alerts, agent_runs Chat Graph reads: all above + users + assessments + exercise_logs + chat_messages Chat Graph writes: chat_messages, alerts

Assessment Graph → Database

WRITES
health_snapshots frailty_evaluations care_plans alerts agent_runs

Chat Graph ← Database

READS (build_user_context)
users health_snapshots assessments frailty_evaluations care_plans exercise_logs chat_messages alerts

Chat Graph → Database

WRITES
chat_messages alerts

Key Constraint

Management sub-agents have no direct DB access. They receive a UserContext dataclass assembled by context_assembly_node and return plain text.

SilverGait · DBA5102 · NUS MSBA