storkit/.living_spec/specs/functional/AI_INTEGRATION.md

Functional Spec: AI Integration

1. Provider Abstraction

The system uses a pluggable architecture for LLMs. The ModelProvider interface abstracts:

• Generation: Sending prompt + history + tools to the model.
• Parsing: Extracting text content vs. tool calls from the raw response.

2. Ollama Implementation

• Endpoint: http://localhost:11434/api/chat
• JSON Protocol:
  • Request: { model: "name", messages: [...], stream: false, tools: [...] }
  • Response: Standard Ollama JSON with message.tool_calls.
• Fallback: If the specific local model doesn't support native tool calling, we may need a fallback system prompt approach, but for this story, we assume a tool-capable model (like llama3.1 or mistral-nemo).

3. Chat Loop (Backend)

The chat command acts as the Agent Loop:

1. Frontend sends: User Message.
2. Backend appends to SessionState.history.
3. Backend calls OllamaProvider.
4. If Text Response: Return text to Frontend.
5. If Tool Call:
   • Backend executes the Tool (using the Core Tools from Story #2).
   • Backend appends ToolResult to history.
   • Backend re-prompts Ollama with the new history (recursion).
   • Repeat until Text Response or Max Turns reached.

4. Frontend State

• Settings: Store llm_provider ("ollama"), ollama_model ("llama3.2"), ollama_base_url.
• Chat: Display the conversation. Tool calls should be visible as "System Events" (e.g., collapsed accordions).