> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Export a trained model
> Publish an already-trained model to Hugging Face, GGUF, or Ollama without re-running training
Publish a model you already trained to Hugging Face, GGUF, or Ollama — no re-training, one command.
```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
subgraph "Export a trained model"
M[lora_model/] --> T{Target?}
T -->|hf| HF[Hugging Face
merged 16-bit]
T -->|gguf| GG[GGUF
quantized]
T -->|ollama| OL[Ollama
ollama.com]
end
classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
classDef output fill:#10B981,stroke:#7C90A0,color:#fff
class M input
class T question
class HF,GG,OL output
```
**Discover the model-dir first.** List the checkpoints a run saved, pick a step, then point `--model-dir` at it — no `ls` needed.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
praisonai-train checkpoints -d outputs --json | jq -r '.[0].path'
praisonai-train export ollama --model-dir outputs/checkpoint-1000 --ollama me/my-model
```
See [List Checkpoints](/docs/features/praisonai-train-checkpoints).
## Quick Start
You already have a trained model in `lora_model/`. Pick a target and publish.
Merge and push FP16 weights to a Hub repo.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
praisonai-train export hf --model-dir lora_model --hf me/my-model
```
Write a local `.gguf` — add `--hf` to also push it to the Hub.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# Local .gguf only
praisonai-train export gguf --model-dir lora_model --quant q4_k_m
# Local .gguf + push to the Hub
praisonai-train export gguf --model-dir lora_model --hf me/my-model --quant q4_k_m
```
Create and push a quantized model to `ollama.com`.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
praisonai-train export ollama --model-dir lora_model --ollama me/my-model --quant q4_k_m
```
**Ollama export ≤ PR #4239 shipped an indented `TEMPLATE`.** The Modelfile embedded four leading spaces on every continuation line, so the served prompt format tokenised differently from training and degraded fine-tune quality. Re-export any Ollama model built before this fix — the export itself takes seconds and does not re-train. Fixed in [PraisonAI PR #4239](https://github.com/MervinPraison/PraisonAI/pull/4239).
***
## When to use this
You already trained a model (e.g. `lora_model/` exists) and don't want to spend an hour re-running training just to publish it to a different target, fix a `chat_template`, or try a different quantization. `export` loads the trained weights once and publishes them.
***
## How It Works
```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
Cfg[--model-dir + flags] --> Load[TrainModel.for_export]
Load --> Base[Infer base model
for chat template]
Base --> Pub{Target}
Pub -->|hf| M[save_model_merged]
Pub -->|gguf| G[save_model_gguf
+ optional push]
Pub -->|ollama| O[create_and_push_ollama_model]
classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
classDef question fill:#F59E0B,stroke:#7C90A0,color:#fff
classDef output fill:#10B981,stroke:#7C90A0,color:#fff
class Cfg input
class Load,Base process
class Pub question
class M,G,O output
```
Export skips the training-only preflight, so it runs on CPU — there's nothing to fine-tune. The base model name drives chat-template selection so the exported model isn't garbled.
***
## CLI flags
| Flag | Short | Purpose | Default |
| --------------------- | ----- | ------------------------------------------------------------------ | -------- |
| `TARGET` (positional) | — | `ollama` \| `gguf` \| `hf` | required |
| `--model-dir` | `-d` | Directory of the already-trained model | required |
| `--config` | `-c` | Optional `config.yaml` for extra export knobs | — |
| `--ollama` | — | Ollama model name, e.g. `me/my-model` | — |
| `--hf` | — | Hugging Face repo id, e.g. `me/my-model` | — |
| `--quant` | — | Quantization method for gguf/ollama, e.g. `q4_k_m` | — |
| `--base-model` | — | Base model id (for chat-template selection) | inferred |
| `--mtp-draft` | — | Also fetch the stock MTP drafter (Gemma-4 only) for fast inference | `false` |
`gguf` always writes a **local** `.gguf` (so you can serve it). Add `--hf me/my-model` to additionally push it to the Hub. Without `--hf`, the GGUF is written under `/gguf`.
***
## How the base model is inferred
The base model name selects the chat template, so the exported model formats prompts correctly. Resolution order:
1. `--base-model` if passed.
2. else `model_name` in `--config`.
3. else `_name_or_path` from `/config.json`.
4. else the model-dir name itself.
***
## Prerequisites
`HF_TOKEN` env var **or** a cached `huggingface-cli login`. The token must have **write** scope and the repo must be under your own username.
```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
export HF_TOKEN=hf_...
# or
huggingface-cli login
```
* `ollama` CLI installed (from [https://ollama.com](https://ollama.com)).
* Your signing key registered at [https://ollama.com/settings/keys](https://ollama.com/settings/keys) — public key lives at `~/.ollama/id_ed25519.pub`.
* Enough free space on `$OLLAMA_MODELS` (defaults to `~/.ollama/models`).
CUDA is **not** required for export-only. CPU is fine because there is nothing to fine-tune.
***
## Valid `--quant` values
Any of these 35 quantization methods (from `VALID_QUANTIZATION_METHODS` — 24 standard plus 11 imatrix).
### Standard (24)
```
bf16 f16 f32 fast_quantized not_quantized quantized
q2_k q2_k_l q3_k_l q3_k_m q3_k_s q3_k_xs
q4_0 q4_1 q4_k q4_k_m q4_k_s
q5_0 q5_1 q5_k q5_k_m q5_k_s
q6_k q8_0
```
### IMatrix — large-model / laptop-fit (11)
Importance-matrix quants let a 30B model fit on a laptop. Unsloth resolves the matrix itself, downloading it from `unsloth/-GGUF` when none is supplied.
```
iq1_m iq1_s iq2_m iq2_s iq2_xs iq2_xxs
iq3_m iq3_s iq3_xxs iq4_nl iq4_xs
```
A typo like `q4km` fails fast with the full list of valid choices — no long run wasted.
See [Unsloth Surface](/docs/features/praisonai-train-unsloth-surface#expanded-quant-list) for the "which quant should I pick" decision guide.
***
## Failure modes → clean errors
Every known failure prints a single actionable message instead of a raw traceback.
| You'll see | What happened | Fix |
| ----------------------------------------------------------------------------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `Hugging Face rejected the token (401)...` | HF token invalid/expired | `huggingface-cli login` or `export HF_TOKEN=hf_...` with a **write** token |
| `No write access to '' (403)...` | Repo not under your username, or token lacks write scope | Set `--hf me/my-model` (your username); regenerate a write token |
| `Ollama rejected the push (unauthorized)...` (your public key printed inline) | Ollama signing key not registered | Add the printed `~/.ollama/id_ed25519.pub` key at [https://ollama.com/settings/keys](https://ollama.com/settings/keys) |
| `Not enough disk for the Ollama model: X GB free on '...', need ~Y GB` | `$OLLAMA_MODELS` volume too small | `export OLLAMA_MODELS=/big/disk/ollama-models` |
| `ollama serve did not become ready in 15 seconds. ollama serve output: ...` | Daemon failed to start (port busy, bad `OLLAMA_HOST`) | Inspect the stderr shown in the message |
| `quantization_method 'q4km' is not valid. Choose one of: ...` | Typo in `--quant` | Pick a value from the list above |
The readiness probe honours `OLLAMA_HOST` and confirms `GET /api/version` — a healthy remote or non-default daemon is used instead of being ignored.
**401 / 403 translation now covers every push path.** As of PraisonAI [PR #4357](https://github.com/MervinPraison/PraisonAI/pull/4357), the same actionable 401/403 messages are raised by the LLM path, **vision training**, and `praisonai-train upload-vision` — previously only the LLM path translated them and the vision paths surfaced a raw `HfHubHTTPError` traceback. See [Hub Privacy & Upload Options](/docs/features/praisonai-train-hub-privacy#error-translation).
**Plain vision runs no longer publish (PraisonAI [#4879](https://github.com/MervinPraison/PraisonAI/pull/4879)).** A local vision fine-tune that omits `huggingface_save` used to push to the Hub — the flag defaulted to the string `"true"` — or crash with `KeyError: 'hf_model_name'` after training completed. From #4879 the vision path skips every export stage unless the flag **and** its target (`hf_model_name` / `ollama_model`) are both set, matching the LLM trainer. See [Vision Fine-Tuning](/docs/features/praisonai-train-vision).
***
## Python API
Script exports with the `TrainModel.for_export()` classmethod — it builds an export-only trainer (no dataset required) and validates the quantization up front.
```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonai_train.train.llm.trainer import TrainModel
trainer = TrainModel.for_export({
"final_model_dir": "lora_model",
"model_name": "unsloth/gemma-2-2b-it-bnb-4bit",
"hf_model_name": "me/my-model",
"quantization_method": "q4_k_m",
})
model, tokenizer = trainer.load_model()
trainer.model, trainer.hf_tokenizer = model, tokenizer
trainer.save_model_merged() # Hugging Face merged 16-bit
# trainer.save_model_gguf() # local GGUF
# trainer.push_model_gguf() # push GGUF to the Hub
# trainer.create_and_push_ollama_model() # Ollama
```
### Hub push options
These keys flow through `config.yaml` (not CLI flags today) into every `push_to_hub_merged` / `push_to_hub_gguf` call. Pass them in the `--config` file or the `for_export({...})` dict.
| Key | Type | Default | Description |
| ---------------- | ------------- | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `hf_private` | bool | `true` | Push as a **private** repo. Set `false` to publish publicly. |
| `save_method` | string | `"merged_16bit"` | Passed to Unsloth's `push_to_hub_merged`. |
| `commit_message` | string | — | Optional Hub commit message. Only sent when set. |
| `tags` | list\[string] | — | Optional Hub repo tags. Only sent when set. |
| `imatrix_file` | str | — | Path to a precomputed importance matrix; forwarded to `push_to_hub_gguf`. Only sent when set — Unsloth auto-fetches one from `unsloth/-GGUF` otherwise. |
Every push is now **private by default**. Add `hf_private: false` to publish publicly — see [Hub Privacy & Upload Options](/docs/features/praisonai-train-hub-privacy).
***
## Best Practices
Set `--hf me/my-model` (not a bare name). A namespaced repo id is treated as a Hub target, never a local directory to delete.
Pass `--quant q4_k_m` so the model is packed small. Ollama exports run `ollama create --quantize`, avoiding a huge FP16 intermediate.
If the exported model's output looks garbled, pass `--base-model` explicitly (e.g. `unsloth/gemma-2-2b-it-bnb-4bit`) so the right chat template is used.
***
## Related
List what a run saved, then export the step you pick.
Prompt a model you just trained and watch it stream.
Full fine-tuning flow and config.yaml reference.
Every `praisonai train` subcommand, including `export`.
Install matrix for the standalone trainer.
Fine-tune across every GPU with torchrun.
Private-by-default pushes, save method, commit message, and tags.
Model access, PEFT selectors, offline merge, and the full 35-quant list.