Production

LLM deployment

From model weights to a callable service. Product pricing with an adapted Llama model on Modal.

ModalLlamaPEFT

Try the experience

LOCAL PROTOTYPE

Illustrative sample data and scripted interactions. No live models, API calls, or notifications. These examples show the proposed experience, not measured model results.

CHOOSE A SAMPLE PRODUCT

Over-ear wireless headphones with active noise cancellation, a travel case, and 30-hour battery life.

Switch products to explore another prepared example.

A model, behind an interface.

Choose a product and reveal its sample result.

UNDER THE HOOD

How it works

The Python service loads Llama 3.2 3B with a PEFT adapter and 4-bit quantisation. A Modal class keeps model loading separate from inference, with a volume for the Hugging Face cache.

01Product description
02Modal GPU service
03Llama + adapter
04Price estimate

A design decision

Allow the GPU service to scale to zero when idle. This reduces idle compute, with a startup wait for the next visitor. A saved example keeps the project explorable while the model starts.