Models & inference · Go
Run open models locally with a single command and a simple local API.
Search prompts, categories, platforms and repositories
vLLM is the default choice for serving open models at scale. PagedAttention improves memory efficiency, continuous batching raises throughput, and the OpenAI-compatible server makes migration straightforward. It is a server-side tool; GPU capacity planning and model compatibility should be validated before production.
Commands are copied from the project documentation at the time of review. Check the repository for the current version.
pip install vllm vllm serve meta-llama/Llama-3.1-8B-Instruct
vllm is a strong default for models & inference, but the right choice depends on your stack, licence constraints and how much control you need. These are the projects we compare most often.
Design a rate limiter
Design a rate limiter with a prompt built for Claude and ChatGPT. It supplies context, constraints and an output contract, then asks the model to surface assumptions and gaps. Copy it, fill the variables and treat the result as a structured draft, not a final answer.
Design an agent cost model
This copy-ready agents prompt helps you design an agent cost model with fewer blank-page decisions. Paste it into Claude and ChatGPT, fill the variables, and you get a structured first draft you can edit. It is written for repeatable results and tells you what to check before publishing.
Write an agent observability plan
A agents prompt for write an agent observability plan. It gives Claude and ChatGPT a clear role, explicit constraints and a defined output, so the first answer is closer to usable. Fill in the context, run it once, then use the verification notes on this page to review the result.
Adoption questions we hear most often about this project.
Models & inference · Go
Run open models locally with a single command and a simple local API.
Models & inference · Python
The reference library for state-of-the-art pretrained models across text, vision and audio.
Models & inference · C++
Efficient C/C++ inference for Llama-family models on CPUs, GPUs and edge devices.