Skip to content
Menu

Everything you need for local inference.

Monitoring. One-click recipe run. Inference proxy. Security. Usage control. Performance benchmark. Cluster management. Storage management. Auto restart on failure. Auto update.

From one GPU to a server farm… From one user to a crowd… From one model to hundreds…

What's giving you trouble?

Finding reliable recipesManaging GPUs and users

So many ingredients. One kitchen.

  • NVIDIA GPUs
  • vLLM
  • Qwen
  • Hugging Face
  • OpenAI
  • SGLang
  • Docker
  • DeepSeek
  • bartowski
  • Unsloth
  • mradermacher
  • llama.cpp
  • Anthropic
  • CUDA
  • GLM
  • Ollama
  • OpenRouter
  • PyTorch
  • ModelScope
  • TheBloke
  • LM Studio Community
How it works

LLM inference is hard.

Find your recipe: GPU, Model, Engine, Params…

GPUs

RTX 3090RTX 4090RTX 50902 × RTX 6000A100H100H200B200B3001 GPU?2 GPUs?4 GPUs?8 GPUs?

Models

QwenDeepSeekGLMQwen2.5 7BDeepSeek · reasoningGLM · chatQwen · visionWhich size?Which quantization?

Engines

vLLMSGLangllama.cppvLLMNightly build?Which CUDA version?vLLMSGLangllama.cpp

Parameters

BF16FP8INT4AWQGPTQFP1632k context128k contextTP 1TP 48k contextbatch sizeKV cacheGPU memorymax tokensTP 2dtypemax sequenceschunked prefillparallelismattention backendCPU offloadcache dtypetoken budgeteager modeCUDA graphs…
Recipe
2 × RTX 6000
Qwen2.5 7B
vLLM
FP168k contextTP 2

…then test it by hand, again and again.

With GPU Kitchen, start from a template…

…and try every variation at once. Recipe Builder

Each one tasted automatically on real workloads. Auto explore

A relaxed administrator at his laptop

Recipes that work, served on your GPUs.Not magic. Just a lot of careful tasting.

Running models is hard.

GPU Kitchen handles it.

Production that runs itself.

  • Which model runs where?Dashboards Live load, waiting requests and capacity of every model.
  • GPU 3 stopped respondingGPU health Spot a faulty GPU and bring it back in one click.
  • The chat model crashed at 3 amCrash recovery Models come back on their own after a crash or a reboot.
  • Everything is slow since 9 amManage priority A fair share of GPU time for everyone when GPUs are busy.
  • Can my team get Llama?On-demand models A model starts on its first request and unloads when idle.
  • Who used all the GPUs?Usage Who uses which model, and how much.
Your GPU Kitchen installationUsers on the left, your controller and proxy in the middle, GPU servers on the right. Recipes combine a model, an engine, GPU requirements and parameters. The proxy routes requests, streams responses and continuously measures live traffic. Monitor illustrates temperature and load on every GPU. Co-operate shows independent users sharing public benchmarks, recipe reliability and error reports through their controllers.INTERNETGPU.Kitchen hubRecipes & metricsHugging FaceModel weightsDocker registryEngine imagesYOUR INFRASTRUCTURECONTROLLER NODENode 01GPU Kitchen workerNode 02GPU Kitchen workerNode 03GPU Kitchen workerGPU KitchencontrollerDownload & installLaunch GPU Kitchen ✓Detect & check GPUsHardware ready ✓AdministratorChatConversationsCodeDeveloper toolsAppsYour applicationsInference proxyOne OpenAI-compatible endpointConfiguration flows down to the proxy. Live monitoring flows back to the controller.ConfigLive monitoringGPU 1GPU 2GPU 3GPU 4Node 1 GPU 1Node 1 GPU 2Node 1 GPU 3Node 1 GPU 4Node 2 GPU 1Node 2 GPU 2Node 2 GPU 3Node 2 GPU 4Node 3 GPU 1Node 3 GPU 2Node 3 GPU 3Node 3 GPU 4Qwen 32BQwen 3BQwen 32BQwen 32BModel filesChat · KV cache kept ↺Code & Apps · KV cache kept ↺TemperatureLoad · liveRecipe errorRecipe reliability · hourlyPublic-model benchmarksRecipe errors · diagnosticsFind your recipeQwen✓ Matched to your GPUsQwen 32BMODELQwen 32BENGINEvLLMGPUs2 × GPUPARAMSTP 2 · FP8Qwen 3BMODELQwen 3BENGINEvLLMGPUs1 × GPUPARAMSTP 1 · FP8Run recipe →Model weights0%vLLM image0%✓ vLLM image already availableKitchen HubRecipes & Metrics

01 / Install

Start on your hardware.

One command installs GPU Kitchen on your Linux host.

02 / Run

Choose a recipe. Run it.

Run recipes matched to your hardware in one click. GPU Kitchen fetches the required model and engine images.

03 / Scale

More nodes. Same workflow.

Grow from one host to hundreds of nodes. Install GPU Kitchen worker on each Linux server and manage the whole fleet from your controller.

Your controller can run without GPUs; workers handle the models.

04 / Serve

One endpoint. Every model.

Serve your users and applications through one fast inference proxy.

One OpenAI-compatible endpoint. Each request reaches the model your application selects.

05 / Monitor

Know how it is running.

See hardware health and quality of service in real time. Track temperature and load on every GPU, alongside memory, throughput and response latency.

06 / Lab

Get more from every GPU.

A complete inference lab. Run parameter sweeps across inference engines, collect benchmarks and compare thousands of recipes to get the most from every GPU.

Results from public models and engines are pooled in the GPU.Kitchen hub, so everyone benefits from the research.

07 / Co-operate

Better recipes, together.

Every hour, GPU Kitchen shares reliability measurements for recipes running public models. Benchmarks using public models and engines also contribute to the GPU.Kitchen hub.

Successful runs, recipe crashes and redacted error reports help everyone choose recipes that work.

Discover the community ↗

Enterprise ready.

Your infrastructure. Your security policies. Your data.

Security comes first.

  • Hardened installation profiles. The Managed network profile enforces sign-in and administrator control over node admission.
  • Protected communications. Mutual TLS secures worker control links. Deploy TLS at your inference endpoint.

Your team. Your policies.

Connect your identity provider with LDAP or OIDC. Manage access by user and team, choose who goes first when the GPUs are full, and cap spending on paid AI services.

Fast, resilient inference.

A high-performance Rust proxy built for demanding industrial workloads. Redundant controllers and proxy replicas keep inference serving during controller failover.

Local inference. Clear boundaries.

Inference stays entirely on your infrastructure. Your prompts and responses never go to the GPU.Kitchen hub.

Proven foundations.

Docker runs your engines. PostgreSQL stores your configuration. Rust powers the worker and proxy.

  • Docker
  • PostgreSQL
  • Rust
In development

Roadmap

  • Public benchmark leaderboards
  • More inference workloads, including industrial applications
Install GPU Kitchen ↑Explore enterprise ↗

Transparent by design.

Inference stays local, and prompts and responses never reach the hub. These are the five channels through which your installation contributes to the GPU.Kitchen hub.

GPU inventory
Installation, edition, GPU models and counts per node.
Public benchmarks
Measurements, recipe parameters and hardware when both model and engine are public.
Recipe reliability
Public-recipe launches and crashes, plus hourly aggregates of uptime, load, performance and errors.
Private-model launches
Host name, engine and GPU information. No model identity, recipe identity, logs or benchmark results.
Technical errors
Redacted diagnostics, limited application stack frames and deduplicated occurrence counts.

Before registration, an installation only checks for updates and, during the beta, checks the time. Once your organization registers it with a GPU Kitchen account, these channels are on by default. Studio and Enterprise installations can turn each one off. Publishing a recipe in the catalog remains a separate action. Privacy policy