Repository navigation
[Feature] Work as an HTTP service. #1028
Description
Feature Summary
Work as an HTTP service.
Detailed Description
Thanks for building this project, this is a fast and clean implementation.
I'd like to suggest adding a server API mode to help with workflows that involve generating many images with different prompts and settings.
Right now with the sd executable, every run reloads the model, which has much overhead on loading models when inferring with variations. A persistent server would allow:
- Loading the model once, then handling multiple generation task via API
- Rapid iteration over different prompts, sizes, steps, etc. without the relaunching overhead
- Easier integration into apps, web UIs, or as a tool for LLM agents
This would make the project much more practical for my personal use case.
Alternatives you considered
No response
Additional context
No response
Reacted by xxnuo
Activity
TkskKurumi commentedon Dec 2, 2025
I have considered that the following difficulties might exist, so I understand that development could be challenging. I feel bad for not having sufficient development skills to contribute directly. I realize I'm just making suggestions from the sidelines here.
Improved exception handling needed? (I noticed in convert to a rest service api, rather than a cli tool #674 (comment) that server feature is already requested and stduhpf forked his own implementation, but but mentioned: “It's a bit annoying because if anything goes wrong during inference, the server kills itself.” Yes, stduhpf said stduhpf have switched to another project, kobold.cpp and not quite activately work on the fork. So I think for long-running inference services, exception handling might be a tricky issue. kobold.cpp doesn’t meet my needs—that project is too bloated and doesn’t yet support ZImage. This project is image/video-generation-focused and implemented ZImage support very early; ZImage was only released a few days ago and already has support here. I'm really amazed and appreciate that.)
I speculate that when certain settings change, the computation buffer size may change and need reallocation, and other preprocessing optimization techniques might become invalid. I don’t fully understand the specifics—just guessing.
Green-Sky commentedon Dec 2, 2025
The most up-to-date pr that provides an http server is #367
There also was #221 .
https://github.com/LostRuins/koboldcpp also provides an http interface.
I think there where more too...
dsignarius commentedon Dec 2, 2025
I would echo the suggestion to try Koboldcpp
The api functions well and provides all sides, together with web gui for both text and image.
The z-image is being worked on, it will not be long.