feat: Backend cancellation support for interrupting model responses
Merged from feature/interrupt-on-type branch. Backend cancellation infrastructure: - Added tokio watch channel to SessionState for cancellation signaling - Implemented cancel_chat command - Modified chat command to use tokio::select! for racing requests vs cancellation - When cancelled, HTTP request to Ollama is dropped and returns early - Added tokio dependency with sync feature Story updates: - Story 13: Updated to use Stop button pattern (industry standard) - Story 18: Created placeholder for streaming responses - Stories 15-17: Placeholders for future features Frontend changes: - Removed auto-interrupt on typing behavior (too confusing) - Backend infrastructure ready for Stop button implementation Note: Story 13 UI (Stop button) not yet implemented - backend ready
This commit is contained in:
@@ -189,3 +189,42 @@ The chat input field should automatically receive focus when the chat component
|
||||
|
||||
return <input ref={inputRef} ... />
|
||||
```
|
||||
|
||||
## Response Interruption
|
||||
|
||||
### Problem
|
||||
Users may want to interrupt a long-running model response to ask a different question or change direction. Having to wait for the full response to complete creates friction and wastes time.
|
||||
|
||||
### Solution: Interrupt on Typing
|
||||
When the user starts typing in the input field while the model is generating a response, the generation should be cancelled immediately, allowing the user to send a new message.
|
||||
|
||||
### Requirements
|
||||
|
||||
1. **Input Always Enabled:** The input field should remain enabled and usable even while the model is generating
|
||||
2. **Interrupt Detection:** Detect when user types in the input field while `loading` state is true
|
||||
3. **Immediate Cancellation:** Cancel the ongoing generation as soon as typing is detected
|
||||
4. **Preserve Partial Response:** Any partial response generated before interruption should remain visible in the chat
|
||||
5. **State Reset:** UI should return to normal state (ready to send) after interruption
|
||||
6. **Preserve User Input:** The user's new input should be preserved in the input field
|
||||
7. **Visual Feedback:** "Thinking..." indicator should disappear when generation is interrupted
|
||||
|
||||
### Implementation Notes
|
||||
* Do NOT disable the input field during loading
|
||||
* Listen for input changes while `loading` is true
|
||||
* When user types during loading, call backend to cancel generation (if possible) or just stop waiting
|
||||
* Set `loading` state to false immediately when typing detected
|
||||
* Backend may need a `cancel_chat` command or similar
|
||||
* Consider if Ollama requests can be cancelled mid-generation or if we just stop processing the response
|
||||
* Example implementation:
|
||||
```tsx
|
||||
const handleInputChange = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||
const newValue = e.target.value;
|
||||
setInput(newValue);
|
||||
|
||||
// If user starts typing while model is generating, interrupt
|
||||
if (loading && newValue.length > input.length) {
|
||||
setLoading(false);
|
||||
// Optionally call backend to cancel: invoke("cancel_chat")
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user