> ## Documentation Index
> Fetch the complete documentation index at: https://praison.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Model access, PEFT selectors & offline merge

> Gated models, custom-code models, MoE LoRA, offline merged export, and vLLM rollouts for GRPO

Reach gated and custom-code models, LoRA only the layers you want, merge offline into a folder, and turn on fast vLLM rollouts for GRPO — all from `config.yaml`.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Unsloth Surface"
        C[📋 config.yaml] --> A[🔑 Model access]
        C --> P[🧩 PEFT selectors]
        C --> M[💾 Offline merge]
        C --> V[💨 vLLM rollouts]
        C --> Q[📦 Quant]
        A --> T[✅ Trained model]
        P --> T
        M --> T
        V --> T
        Q --> T
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class C input
    class A,P,M,V,Q process
    class T output
```

These keys were always accepted by Unsloth but never forwarded — set any of them today and the run does exactly what you asked. All are additive; existing configs keep working.

## Quick Start

Train a gated Llama base with a token, pin the revision, and merge offline — no Hub account needed.

**1. Start from an Agent**

Pick the model you want to fine-tune, then reach it from a config.

```python theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
from praisonaiagents import Agent

agent = Agent(instructions="Answer briefly and correctly.")
agent.start("What is the capital of France?")
```

**2. Reach a gated base and pin a revision**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# config.yaml
model_name: "meta-llama/Meta-Llama-3.1-8B-Instruct"
max_seq_length: 2048
hf_token: "hf_xxx"          # or leave unset and use HF_TOKEN
revision: "main"            # pin so a multi-day run is reproducible
trust_remote_code: false
dataset:
  - name: "yahma/alpaca-cleaned"
```

**3. Merge into a folder, no Hub**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
merged_save_dir: "./merged-model"
save_method: "merged_16bit"   # default
```

```bash theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
pip install "praisonai-train[llm]"
praisonai-train llm config.yaml
```

***

## Model access

`hf_token`, `trust_remote_code`, and `revision` decide **which** weights load and how they are reached.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Model access"
        Cfg[📋 config] --> Tok{gated repo?}
        Tok -->|yes| HT[🔑 hf_token → token]
        Tok -->|no| Skip[skip]
        Cfg --> TR[🧠 trust_remote_code]
        Cfg --> Rev[📌 revision pin]
        HT --> Load[✅ FastLanguageModel]
        TR --> Load
        Rev --> Load
    end

    classDef input fill:#6366F1,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef process fill:#189AB4,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class Cfg input
    class Tok decision
    class HT,TR,Rev,Skip process
    class Load output
```

**1. Gated model (Llama / Gemma)**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
# config.yaml
model_name: "meta-llama/Meta-Llama-3.1-8B-Instruct"
max_seq_length: 2048
hf_token: "hf_xxx"          # config-level equivalent of HF_TOKEN
dataset:
  - name: "yahma/alpaca-cleaned"
```

**2. Custom-code model**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "some-org/custom-arch-model"
trust_remote_code: true      # required to run the repo's model code
```

**3. Pin a revision for reproducibility**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "meta-llama/Meta-Llama-3.1-8B-Instruct"
revision: "main"             # a branch, tag, or commit SHA
```

| Key | Type | Default | Description |
| - | - | - | - |
| `hf_token` | `str` | — | Forwarded to Unsloth as `token`. Renamed so it can't collide with the trainer's own token handling. Only sent when set. |
| `trust_remote_code` | `bool` | — | Required for custom-code models. Only sent when set. |
| `revision` | `str` | — | Pin a HF branch, tag, or SHA for a reproducible multi-day run. Only sent when set. |

> [!NOTE]
>
> Config `hf_token` is the config-level twin of the `HF_TOKEN` env var. Precedence: `HF_TOKEN` env var wins, then a cached `huggingface-cli login`, then config `hf_token`. See [Hub Privacy & Upload Options](https://praison.ai/docs/features/praisonai-train-hub-privacy.md).

***

## PEFT selectors

Choose exactly which layers and parameters LoRA touches — from the last N transformer blocks to a MoE model's experts.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start["📋 How much can I spend?"] --> Q{Budget?}
    Q -->|"full quality"| Full["⚙️ full LoRA<br/>all target modules"]
    Q -->|"cheap run"| Last["⚙️ finetune_last_n_layers<br/>last N blocks only"]
    Q -->|"MoE model"| Moe["⚙️ target_parameters<br/>expert adapters"]

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef method fill:#6366F1,stroke:#7C90A0,color:#fff

    class Start input
    class Q decision
    class Full,Last,Moe method
```

**1. Cheap run — last N layers**

Tune only the last few transformer blocks to cut memory and time.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "unsloth/Qwen2.5-32B-Instruct-bnb-4bit"
max_seq_length: 2048
finetune_last_n_layers: 8
dataset:
  - name: "yahma/alpaca-cleaned"
```

**2. MoE expert LoRA**

`target_parameters` is what lets LoRA reach a Mixture-of-Experts model's experts.

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "unsloth/mistral-8x7b-instruct-v0.3-bnb-4bit"
max_seq_length: 2048
target_parameters:
  - "mlp.experts"           # selector; confirm the exact name in Unsloth's docs
dataset:
  - name: "yahma/alpaca-cleaned"
```

**3. Explicit layers and init scheme**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
layers_to_transform: [0, 1, 2, 3]     # explicit layer indices
layers_pattern: "model.layers"        # pattern for layer selection
init_lora_weights: "gaussian"         # "gaussian", "pissa", "olora", ...
```

| Key | Type | Default | Description |
| - | - | - | - |
| `finetune_last_n_layers` | `int` | — | Tune only the last N transformer blocks — the cheap-run recipe. |
| `layers_to_transform` | `int \| list[int]` | — | Explicit layer indices to adapt. |
| `layers_pattern` | `str` | — | Regex / pattern selecting which layers to adapt. |
| `target_parameters` | `list[str]` | — | Parameter selectors — required to LoRA MoE experts. |
| `init_lora_weights` | `str \| bool` | — | LoRA init scheme, e.g. `"gaussian"`, `"pissa"`, `"olora"`. |

Each is forwarded to `FastLanguageModel.get_peft_model` only when set, so they compose with the existing `lora_r`, `lora_alpha`, `use_dora`, and `rank_pattern` keys.

***

## Offline merged export

`merged_save_dir` writes merged weights to a folder on this disk — no Hub account, no token, no network.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph LR
    subgraph "Merge routes"
        Ad[🧩 Trained adapter] --> R{Where?}
        R -->|"merged_save_dir"| Loc[💾 save_pretrained_merged<br/>local folder]
        R -->|"hf_model_name"| Hub[🌐 push_to_hub_merged<br/>Hub repo]
    end

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef output fill:#10B981,stroke:#7C90A0,color:#fff

    class Ad input
    class R decision
    class Loc,Hub output
```

**1. Merge into a folder**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "unsloth/gemma-2-2b-it-bnb-4bit"
max_seq_length: 2048
merged_save_dir: "./merged-model"
save_method: "merged_16bit"   # default; also "merged_4bit_forced", "lora"
dataset:
  - name: "yahma/alpaca-cleaned"
```

| Key | Type | Default | Description |
| - | - | - | - |
| `merged_save_dir` | `str` | — | Merge the adapter into this local directory via `save_pretrained_merged`. No-op when unset. |
| `save_method` | `str` | `"merged_16bit"` | Merge precision — `"merged_16bit"`, `"merged_4bit_forced"`, or `"lora"`. |

Every run calls the merge step; it does nothing unless `merged_save_dir` is set. Contrast this with the Hub-push flow in [Export a trained model](https://praison.ai/docs/features/praisonai-train-export.md), which needs a repo id, a write token, and a network.

***

## vLLM fast rollouts (GRPO)

`fast_inference` swaps GRPO's slow HF-generate loop for vLLM rollouts — one line.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
sequenceDiagram
    participant TRL as GRPOTrainer
    participant Unsloth
    participant Gen as Rollout
    TRL->>Unsloth: generate completions
    Unsloth->>Gen: fast_inference? 
    Gen-->>Unsloth: vLLM (fast) or HF-generate (slow)
    Unsloth-->>TRL: scored completions
    TRL->>TRL: gradient update
```

**1. Turn on fast rollouts**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
method: grpo
model_name: "unsloth/gemma-2-2b-it-bnb-4bit"
max_seq_length: 2048
fast_inference: true
gpu_memory_utilization: 0.6
max_lora_rank: 32
num_generations: 8
reward_funcs:
  - myproject.rewards:length_penalty
dataset:
  - name: "your-org/your-prompts"
```

| Key | Type | Default | Description |
| - | - | - | - |
| `fast_inference` | `bool` | `false` | Turn on vLLM rollouts for GRPO. GRPO previously always ran the slow HF-generate path. |
| `gpu_memory_utilization` | `float` | — | vLLM GPU memory fraction. Only forwarded when `fast_inference` is truthy. |
| `max_lora_rank` | `int` | — | Max LoRA rank vLLM allocates for. Only forwarded when `fast_inference` is truthy. |

> [!NOTE]
>
> `gpu_memory_utilization` and `max_lora_rank` are ignored unless `fast_inference` is on. See [Reward Functions & GRPO](https://praison.ai/docs/features/train-reward-functions.md).

***

## Expanded quant list

Unsloth's exporter accepts 35 quantization methods — 24 standard plus 11 importance-matrix (IQ) quants. The IQ family is what lets a 30B model fit on a laptop.

```mermaid theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
graph TB
    Start["📋 Which quant?"] --> Q{Priority?}
    Q -->|"balanced"| Std["📦 q4_k_m<br/>standard"]
    Q -->|"best quality"| Big["📦 q8_0<br/>large, accurate"]
    Q -->|"fit on a laptop"| IQ["📦 iq4_xs / iq3_m<br/>imatrix"]

    classDef input fill:#8B0000,stroke:#7C90A0,color:#fff
    classDef decision fill:#F59E0B,stroke:#7C90A0,color:#fff
    classDef out fill:#10B981,stroke:#7C90A0,color:#fff

    class Start input
    class Q decision
    class Std,Big,IQ out
```

### Standard (24)

```
bf16  f16  f32  fast_quantized  not_quantized  quantized
q2_k  q2_k_l  q3_k_l  q3_k_m  q3_k_s  q3_k_xs
q4_0  q4_1  q4_k  q4_k_m  q4_k_s
q5_0  q5_1  q5_k  q5_k_m  q5_k_s
q6_k  q8_0
```

### IMatrix — large-model / laptop-fit (11)

```
iq1_m  iq1_s  iq2_m  iq2_s  iq2_xs  iq2_xxs
iq3_m  iq3_s  iq3_xxs  iq4_nl  iq4_xs
```

**1. Imatrix quant — 30B on a laptop**

```yaml theme={"theme":{"light":"vitesse-light","dark":"vitesse-dark"}}
model_name: "unsloth/Qwen2.5-32B-Instruct-bnb-4bit"
max_seq_length: 2048
quantization_method: iq4_xs      # imatrix quant — auto-fetches the matrix
# imatrix_file: "./my.imatrix"   # optional; supply your own
dataset:
  - name: "yahma/alpaca-cleaned"
```

| Key | Type | Default | Description |
| - | - | - | - |
| `quantization_method` | `str` | `"q4_k_m"` | Any of the 35 valid quants above. Validated up front — a typo fails fast. |
| `imatrix_file` | `str` | — | Path to a precomputed importance matrix; forwarded to `push_to_hub_gguf`. Only sent when set — Unsloth otherwise auto-fetches one from `unsloth/<base>-GGUF`. |

Full quant reference lives in [Export a trained model](https://praison.ai/docs/features/praisonai-train-export.md#valid-quant-values).

***

## Best Practices

> [accordion] Prefer config hf\_token inside CI
>
> Config-level `hf_token` keeps the token out of environment inspection in CI. Outside CI, `export HF_TOKEN=hf_...` is simpler — the env var wins over config.

> [accordion] Pin revision for any run longer than a day
>
> A branch moves. Set `revision` to a tag or SHA so a run you resume tomorrow loads the same weights.

> [accordion] Reach for finetune\_last\_n\_layers on big models
>
> Tuning the last 8 blocks of a 32B model is a fraction of the memory of full LoRA — start there, widen only if quality needs it.

> [accordion] Merge offline before you decide to publish
>
> `merged_save_dir` gives you the merged weights on disk with no Hub round-trip. Inspect them, then push separately only if you want to.

> [accordion] Use an imatrix quant to fit a large model on a laptop
>
> `iq4_xs` / `iq3_m` pack a 30B model small enough to run locally. Unsloth fetches the matrix automatically; supply `imatrix_file` only if you built your own.

***

## Related

> [card] [Train](https://praison.ai/docs/train.md)
>
> Full fine-tuning flow and config.yaml reference.

> [card] [Train CLI](https://praison.ai/docs/cli/train.md)
>
> Every `praisonai train` subcommand and config key.

> [card] [Export a trained model](https://praison.ai/docs/features/praisonai-train-export.md)
>
> Publish to HF, GGUF, or Ollama — full 35-quant reference.

> [card] [Hub Privacy & Upload Options](https://praison.ai/docs/features/praisonai-train-hub-privacy.md)
>
> Private-by-default pushes and the `hf_token` precedence ladder.

> [card] [Reward Functions & GRPO](https://praison.ai/docs/features/train-reward-functions.md)
>
> Score completions with Python and turn on vLLM fast rollouts.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.