AI Cloud | Yunion OneCloud

Three core modules

Agent instances, inference instances, and the AI gateway form a complete AI Cloud stack

Agent instances

•Containerized: Linux containers with an Ubuntu desktop—one container, one Agent instance.

•Ready to use: Full Ubuntu desktop and isolated workspace, with a browser and common tools preinstalled.

•Secure isolation: Dedicated storage and network; CPU and memory quotas supported.

•Self-service: Access the desktop in a browser and configure agents, models, and IM channels on your own.

Inference instances

•Containerized: Run LLM stacks such as Ollama and vLLM as standard container instances.

•GPU / NPU: Single GPU, multi-GPU, or shared GPUs with MPS / HAMi partitioning; supports NVIDIA, AMD, and Huawei Ascend.

•Fast model delivery: Model artifacts pre-staged on hosts for sub-second attach; curated open-weight model datasets built in.

•Frameworks: vLLM, Ollama, and ComfyUI for inference, serving, and image generation.

AI gateway

•Unified access: Manage cloud AI gateways, providers, and endpoints in one place for multi-model services.

•API keys: Virtual keys with quotas and scopes for secure calls through the platform AI gateway.

•Smart routing: Match by model id and forward traffic to public providers or local inference deployments.

•Multi-provider: OpenAI, DeepSeek, and other public APIs, working alongside private inference.

Agent + LLM deployment patterns

Mix and match by cost, data sovereignty, and model capability—with the AI gateway for access and routing

No-GPU pattern

Deploy Agents at scale on regular servers; the AI gateway connects them to hosted LLM APIs via tokens.

Benefits:no GPU capital expense; pay only for token usage.

On-premises pattern

Agents on standard servers; GPUs host Ollama / vLLM. The AI gateway routes traffic to private inference on the LAN.

Benefits:data stays on the LAN; no public API token charges.

Hybrid pattern

Use both on-prem inference and online LLM APIs; gateway routing rules send easy prompts to local models and escalate complex ones to the cloud.

Benefits:balance cost against the best available model quality.

Platform strengths

Instant availability

Pre-staged models + containers

Instances ready in seconds

Network and storage isolation

API key quotas and scopes

GPU sharing and partitioning

Smart routing to cut spend

Leading LLM frameworks

Unified gateway APIs