30 lines
1.5 KiB
Markdown
30 lines
1.5 KiB
Markdown
# Functional Spec: AI Integration
|
|
|
|
## 1. Provider Abstraction
|
|
The system uses a pluggable architecture for LLMs. The `ModelProvider` interface abstracts:
|
|
* **Generation:** Sending prompt + history + tools to the model.
|
|
* **Parsing:** Extracting text content vs. tool calls from the raw response.
|
|
|
|
## 2. Ollama Implementation
|
|
* **Endpoint:** `http://localhost:11434/api/chat`
|
|
* **JSON Protocol:**
|
|
* Request: `{ model: "name", messages: [...], stream: false, tools: [...] }`
|
|
* Response: Standard Ollama JSON with `message.tool_calls`.
|
|
* **Fallback:** If the specific local model doesn't support native tool calling, we may need a fallback system prompt approach, but for this story, we assume a tool-capable model (like `llama3.1` or `mistral-nemo`).
|
|
|
|
## 3. Chat Loop (Backend)
|
|
The `chat` command acts as the **Agent Loop**:
|
|
1. Frontend sends: `User Message`.
|
|
2. Backend appends to `SessionState.history`.
|
|
3. Backend calls `OllamaProvider`.
|
|
4. **If Text Response:** Return text to Frontend.
|
|
5. **If Tool Call:**
|
|
* Backend executes the Tool (using the Core Tools from Story #2).
|
|
* Backend appends `ToolResult` to history.
|
|
* Backend *re-prompts* Ollama with the new history (recursion).
|
|
* Repeat until Text Response or Max Turns reached.
|
|
|
|
## 4. Frontend State
|
|
* **Settings:** Store `llm_provider` ("ollama"), `ollama_model` ("llama3.2"), `ollama_base_url`.
|
|
* **Chat:** Display the conversation. Tool calls should be visible as "System Events" (e.g., collapsed accordions).
|