kamidso-back/docs/kamidso/NATIVE_DBGPT_MEMORY.md

Native DB-GPT memory and prompt restore

Kamidso’s Agentic ReAct path now uses DB-GPT 0.8.2 memory and ReAct prompt wiring by default. This is a restore of the native baseline, not a new Kamidso memory product.

Feature flags

Variable	Default	1	0
KAMIDSO_USE_NATIVE_DBGPT_MEMORY	1	HybridMemory + EnhancedShortTermMemory + LongTermMemory (when embeddings work) + persistent MetaDbGptsMessageMemory / MetaDbGptsPlansMemory	Previous path: AgentMemory(gpts_memory=...) fallback ShortTermMemory(5) + in-process DefaultGptsPlansMemory
KAMIDSO_USE_NATIVE_DBGPT_PROMPT	1	Native _REACT_SYSTEM_TEMPLATE (includes {{ question }} and {{ task_progress }}) plus a compact Kamidso identity profile	Previous huge workflow_prompt PromptTemplate bind

Independent combinations:

MEMORY=1 PROMPT=1   full native baseline (default)
MEMORY=1 PROMPT=0   native memory, legacy Kamidso prompt
MEMORY=0 PROMPT=1   native prompt, legacy memory
MEMORY=0 PROMPT=0   previous Kamidso behaviour

Add both to .env.example. Development default is on.

Rollback

KAMIDSO_USE_NATIVE_DBGPT_MEMORY=0
KAMIDSO_USE_NATIVE_DBGPT_PROMPT=0

Restart the backend. No schema rollback is required; gpts_messages / gpts_plans already existed.

On-disk paths

All native-memory files are created under the product data/ root (KAMIDSO_DATA_ROOT / KAMIDSO_STORAGE_PATH, default data):

What	Path
Long-term vector store (Chroma)	data/vectors/chromadb
ReAct operation snapshots	data/runtime/op_snapshots/<conv_id>/
Engine GPTs SQLite (if a file DB is used)	data/meta/kamidso-engine.db

Nothing from this restore is written to the repo root, home directory, or .chroma in the working directory.

What is reused (not rewritten)

• HybridMemory.from_vstore (same pattern as MultiAgents controller.get_or_build_agent_memory)
• EnhancedShortTermMemory / LongTermMemory
• MetaDbGptsMessageMemory / MetaDbGptsPlansMemory and tables gpts_messages / gpts_plans
• Native ReAct ProfileConfig.system_prompt_template
• Existing enable_context_management=True (token budget, not memory)

todowrite remains the UI task list. It is not synced to GptsPlansMemory in this phase. Terminate no longer marks leftover todos completed.

Logs

Native memory:

[memory] native DB-GPT memory enabled
[memory] conversation_id=...
[memory] gpts message memory=MetaDbGptsMessageMemory
[memory] plans memory=MetaDbGptsPlansMemory
[memory] hybrid memory enabled
[memory] long-term memory enabled
[memory] vector store=agent_memory_long_term_...
[memory] vector persist_dir=.../data/vectors/chromadb

If embeddings / vector store cannot start:

Native DB-GPT LongTermMemory disabled: embedding model not configured.

The assistant continues with persistent GptsMemory + default short-term memory. This is not silent.

Prompt metadata (development only: KAMIDSO_ENV=development or KAMIDSO_DEVELOPER_MODE=1):

[prompt] native=True question_present=True task_progress_present=True recent_memories_count=0 observation_present=False conversation_id=... round=1

Organization payload and the full prompt are not written.

Known limitations

1. Native prompt mode keeps tools registered but drops the long Kamidso skill/SQL/report essay. Identity + language remain.
2. ReAct does not write GptsPlan rows; persistent plans storage is wired for native planning, not todowrite sync.
3. Long-term vector memory needs a configured embedding model. Failure logs and falls back; no custom vector engine is added.
4. Native DB-GPT does not redact secrets before long-term persistence.
5. In-process _task_progress lives on the agent instance for the current run. After process restart, context comes back from Gpts messages and long-term memory, not that in-memory list.
6. Structured query-result memory is out of scope (next phase).

Manual checklist

Use a multi-step sales investigation, a follow-up that refers to “that branch”, a backend restart + resume, and a mid-run terminate. Confirm:

• question_present=True each round
• task_progress_present=True after the first completed step
• todos are not all flipped to completed on terminate
• conversation A (sales) does not receive conversation B (HR) state