Cloud models wouldn't let users talk, and wouldn't let us adjust them: how an AI companion chat service self-hosted an open-source model
A companion chat service ran on a cloud model. Conversations were often refused, and the characters couldn't be adjusted. We switched to an open-source model: selection, refusal-behaviour tuning, character and memory design, deployed on the client's own servers.
- Client
- An AI companion chat service: open-source model selection, tuning and self-hosted deployment
- Industry
- AI Native
- Service
- Custom AI systems
- Led by
- Johnny Yang

What we delivered.
- Model
- Open-source model, deployed on the client's own servers
- Tuning
- Model's refusal scope adjusted to the service's needs
- Characters
- Character setup + conversation memory
- Management
- Usage rules + conversation logs
01Cloud models are strong, but they aren't yours
The client runs an AI companion chat service. Users chat with pre-defined characters. Each character needs a consistent personality and has to remember what was said before.
At first they used a cloud model. They quickly hit two problems. First, the cloud vendor has its own usage policy, and a lot of normal companion conversations got refused or answered vaguely. Users felt the character had "suddenly become a different person". Second, the model's behaviour couldn't be adjusted at all. When the vendor changed its policy, the service changed with it.
02Why switch to an open-source model
An open-source model can run on servers you control. The service decides how the model behaves, and the cloud vendor's policy changes don't affect it. The cost: selection, deployment, tuning and operations are all on you. That is the part we did.
03What we did
- Selection: Based on the Chinese conversation ability, response speed and hardware budget the service needed, we tested several open-source models, using the client's own conversation scenarios as the test, then decided which one to use.
- Adjusting the model's refusal behaviour: With an open-source model you can adjust "which situations it should refuse to answer". We tuned this to the client's service scope, so normal companion conversations are no longer blocked by mistake, while the lines the client needs to hold stay in place.
- Character setup and conversation memory: Each character has a fixed personality, way of speaking and background. Conversation memory lets the character remember what the user said before, so conversations stay consistent.
- Self-hosted deployment: The model and inference service run on servers the client controls. Conversation data does not pass through an outside AI vendor.
- Rules and logs: Once the cloud vendor is no longer the gatekeeper, the responsibility falls back on the service. So we added usage rules, age and content management settings, and conversation logs, so when something goes wrong it can be traced.
04What this project proved
For Brightstream, this project proved one thing: we can select, tune and deploy open-source models, and we know how to put the gatekeeping responsibility back in place. We use the same capability on on-premise AI projects in medical and consulting work, where "the data can't leave the company".
05Why we don't name the client
This is the client's service brand. We only share the approach.
06My take
The biggest value of an open-source model is that "you decide how the model behaves". The other side of that is: you are also responsible for the gatekeeping. If you only tune the model and don't add rules and logs, you're leaving the risk with yourself.

