Instructions to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("dalatexcoder/MiniCPM5-2B-5BitL-MLX")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-2B-5BitL-MLX"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "dalatexcoder/MiniCPM5-2B-5BitL-MLX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "dalatexcoder/MiniCPM5-2B-5BitL-MLX"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "dalatexcoder/MiniCPM5-2B-5BitL-MLX"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "dalatexcoder/MiniCPM5-2B-5BitL-MLX",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-2B-5BitL-MLX"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default dalatexcoder/MiniCPM5-2B-5BitL-MLX

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use dalatexcoder/MiniCPM5-2B-5BitL-MLX with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "dalatexcoder/MiniCPM5-2B-5BitL-MLX"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "dalatexcoder/MiniCPM5-2B-5BitL-MLX" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Quantized from openbmb/MiniCPM5-2B. Effective average BPW is around 5.669.

Top 20 most-sensitive layers at 2-bit:
1. model.layers.1.mlp.down_proj sens=1.002e+00 params=12,582,912
2. model.layers.7.mlp.down_proj sens=4.134e-01 params=12,582,912
3. lm_head sens=3.690e-01 params=267,386,880
4. model.layers.1.self_attn.q_proj sens=2.767e-01 params=4,194,304
5. model.layers.0.self_attn.v_proj sens=1.715e-01 params=524,288
6. model.layers.41.mlp.down_proj sens=1.299e-01 params=12,582,912
7. model.layers.1.self_attn.o_proj sens=1.231e-01 params=4,194,304
8. model.layers.4.self_attn.v_proj sens=1.165e-01 params=524,288
9. model.layers.2.self_attn.q_proj sens=9.487e-02 params=4,194,304
10. model.layers.2.self_attn.v_proj sens=9.286e-02 params=524,288
11. model.layers.0.mlp.down_proj sens=8.109e-02 params=12,582,912
12. model.layers.13.self_attn.o_proj sens=7.899e-02 params=4,194,304
13. model.layers.14.self_attn.o_proj sens=7.754e-02 params=4,194,304
14. model.layers.3.self_attn.v_proj sens=7.545e-02 params=524,288
15. model.layers.16.self_attn.o_proj sens=7.483e-02 params=4,194,304
16. model.layers.21.self_attn.o_proj sens=7.397e-02 params=4,194,304
17. model.layers.1.self_attn.v_proj sens=7.267e-02 params=524,288
18. model.layers.5.self_attn.q_proj sens=6.835e-02 params=4,194,304
19. model.layers.5.self_attn.v_proj sens=6.390e-02 params=524,288
20. model.layers.6.mlp.down_proj sens=6.340e-02 params=12,582,912

Top 20 most-sensitive layers at 3-bit:
1. model.layers.7.mlp.down_proj sens=1.488e-01 params=12,582,912
2. model.layers.1.mlp.down_proj sens=8.359e-02 params=12,582,912
3. lm_head sens=7.595e-02 params=267,386,880
4. model.layers.2.self_attn.v_proj sens=4.404e-02 params=524,288
5. model.layers.0.self_attn.v_proj sens=4.262e-02 params=524,288
6. model.layers.3.self_attn.v_proj sens=3.825e-02 params=524,288
7. model.layers.2.self_attn.q_proj sens=3.742e-02 params=4,194,304
8. model.layers.1.self_attn.o_proj sens=3.118e-02 params=4,194,304
9. model.layers.41.mlp.down_proj sens=2.823e-02 params=12,582,912
10. model.layers.1.self_attn.q_proj sens=2.587e-02 params=4,194,304
11. model.layers.5.self_attn.v_proj sens=2.548e-02 params=524,288
12. model.layers.4.self_attn.v_proj sens=2.526e-02 params=524,288
13. model.layers.1.self_attn.v_proj sens=2.486e-02 params=524,288
14. model.layers.5.self_attn.q_proj sens=2.426e-02 params=4,194,304
15. model.layers.14.self_attn.o_proj sens=2.276e-02 params=4,194,304
16. model.layers.12.self_attn.v_proj sens=2.275e-02 params=524,288
17. model.layers.0.mlp.down_proj sens=2.242e-02 params=12,582,912
18. model.layers.13.self_attn.o_proj sens=2.229e-02 params=4,194,304
19. model.layers.6.mlp.down_proj sens=2.124e-02 params=12,582,912
20. model.layers.16.self_attn.o_proj sens=2.064e-02 params=4,194,304

Top 20 most-sensitive layers at 4-bit:
1. model.layers.7.mlp.down_proj sens=4.436e-02 params=12,582,912
2. model.layers.1.mlp.down_proj sens=2.787e-02 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=2.504e-02 params=524,288
4. lm_head sens=1.819e-02 params=267,386,880
5. model.layers.2.self_attn.v_proj sens=1.638e-02 params=524,288
6. model.layers.5.self_attn.q_proj sens=1.346e-02 params=4,194,304
7. model.layers.1.self_attn.o_proj sens=1.188e-02 params=4,194,304
8. model.layers.4.self_attn.v_proj sens=1.160e-02 params=524,288
9. model.layers.1.self_attn.q_proj sens=1.062e-02 params=4,194,304
10. model.layers.2.self_attn.o_proj sens=1.054e-02 params=4,194,304
11. model.layers.3.self_attn.v_proj sens=9.660e-03 params=524,288
12. model.layers.2.self_attn.q_proj sens=9.620e-03 params=4,194,304
13. model.layers.10.mlp.up_proj sens=9.215e-03 params=12,582,912
14. model.layers.0.self_attn.o_proj sens=9.097e-03 params=4,194,304
15. model.layers.1.self_attn.v_proj sens=9.091e-03 params=524,288
16. model.layers.0.mlp.down_proj sens=8.764e-03 params=12,582,912
17. model.layers.5.self_attn.v_proj sens=8.714e-03 params=524,288
18. model.layers.9.self_attn.v_proj sens=8.605e-03 params=524,288
19. model.layers.14.self_attn.o_proj sens=7.816e-03 params=4,194,304
20. model.layers.4.mlp.up_proj sens=7.747e-03 params=12,582,912

Top 20 most-sensitive layers at 5-bit:
1. model.layers.7.mlp.down_proj sens=1.577e-02 params=12,582,912
2. model.layers.1.mlp.down_proj sens=9.463e-03 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=7.656e-03 params=524,288
4. model.layers.2.self_attn.v_proj sens=6.293e-03 params=524,288
5. model.layers.1.self_attn.o_proj sens=6.019e-03 params=4,194,304
6. model.layers.5.mlp.up_proj sens=5.718e-03 params=12,582,912
7. model.layers.1.self_attn.q_proj sens=5.607e-03 params=4,194,304
8. model.layers.3.self_attn.v_proj sens=5.563e-03 params=524,288
9. model.layers.4.mlp.down_proj sens=5.433e-03 params=12,582,912
10. model.layers.2.mlp.gate_proj sens=5.274e-03 params=12,582,912
11. model.layers.0.mlp.down_proj sens=5.241e-03 params=12,582,912
12. model.layers.2.self_attn.o_proj sens=5.067e-03 params=4,194,304
13. model.layers.2.self_attn.q_proj sens=5.060e-03 params=4,194,304
14. model.layers.4.mlp.up_proj sens=4.983e-03 params=12,582,912
15. model.layers.1.self_attn.v_proj sens=4.959e-03 params=524,288
16. model.layers.0.self_attn.o_proj sens=4.954e-03 params=4,194,304
17. model.layers.5.self_attn.v_proj sens=4.817e-03 params=524,288
18. model.layers.0.mlp.gate_proj sens=4.768e-03 params=12,582,912
19. lm_head sens=4.615e-03 params=267,386,880
20. model.layers.2.mlp.up_proj sens=4.561e-03 params=12,582,912

Top 20 most-sensitive layers at 6-bit:
1. model.layers.7.mlp.down_proj sens=6.492e-03 params=12,582,912
2. model.layers.1.mlp.down_proj sens=6.159e-03 params=12,582,912
3. model.layers.0.self_attn.v_proj sens=5.739e-03 params=524,288
4. model.layers.3.self_attn.k_proj sens=5.012e-03 params=524,288
5. model.layers.2.self_attn.v_proj sens=4.739e-03 params=524,288
6. model.layers.0.mlp.up_proj sens=4.594e-03 params=12,582,912
7. model.layers.2.self_attn.q_proj sens=4.409e-03 params=4,194,304
8. model.layers.2.mlp.down_proj sens=4.399e-03 params=12,582,912
9. model.layers.6.mlp.up_proj sens=4.348e-03 params=12,582,912
10. model.layers.1.self_attn.v_proj sens=4.302e-03 params=524,288
11. model.layers.2.self_attn.o_proj sens=4.259e-03 params=4,194,304
12. model.layers.4.self_attn.q_proj sens=4.223e-03 params=4,194,304
13. model.layers.1.mlp.gate_proj sens=4.110e-03 params=12,582,912
14. model.layers.2.self_attn.k_proj sens=4.094e-03 params=524,288
15. model.layers.6.self_attn.q_proj sens=3.977e-03 params=4,194,304
16. model.layers.6.mlp.gate_proj sens=3.973e-03 params=12,582,912
17. model.layers.1.self_attn.o_proj sens=3.940e-03 params=4,194,304
18. model.layers.4.mlp.down_proj sens=3.883e-03 params=12,582,912
19. model.layers.5.mlp.gate_proj sens=3.870e-03 params=12,582,912
20. model.layers.0.self_attn.o_proj sens=3.824e-03 params=4,194,304

Top 20 most-sensitive layers at 8-bit:
1. model.layers.1.self_attn.v_proj sens=4.394e-03 params=524,288
2. model.layers.2.self_attn.v_proj sens=4.088e-03 params=524,288
3. model.layers.4.self_attn.k_proj sens=3.958e-03 params=524,288
4. model.layers.1.mlp.gate_proj sens=3.900e-03 params=12,582,912
5. model.layers.0.mlp.up_proj sens=3.824e-03 params=12,582,912
6. model.layers.0.mlp.down_proj sens=3.799e-03 params=12,582,912
7. model.layers.0.self_attn.k_proj sens=3.778e-03 params=524,288
8. model.layers.0.self_attn.o_proj sens=3.697e-03 params=4,194,304
9. model.layers.4.mlp.up_proj sens=3.644e-03 params=12,582,912
10. model.layers.0.self_attn.v_proj sens=3.592e-03 params=524,288
11. model.layers.4.mlp.gate_proj sens=3.513e-03 params=12,582,912
12. model.layers.5.self_attn.v_proj sens=3.422e-03 params=524,288
13. model.layers.1.mlp.up_proj sens=3.421e-03 params=12,582,912
14. model.layers.3.self_attn.q_proj sens=3.398e-03 params=4,194,304
15. model.layers.6.mlp.gate_proj sens=3.394e-03 params=12,582,912
16. model.layers.4.mlp.down_proj sens=3.328e-03 params=12,582,912
17. model.layers.5.self_attn.q_proj sens=3.321e-03 params=4,194,304
18. model.layers.0.self_attn.q_proj sens=3.279e-03 params=4,194,304
19. model.layers.2.mlp.gate_proj sens=3.275e-03 params=12,582,912
20. model.layers.0.mlp.gate_proj sens=3.252e-03 params=12,582,912

Optimization result: 34 @ 3-bit, 95 @ 4-bit, 84 @ 5-bit, 42 @ 6-bit, 40 @ 8-bit

Downloads last month: 151

Safetensors

Model size

3B params

Tensor type

U32

BF16

MLX

Hardware compatibility

4-bit

Model tree for dalatexcoder/MiniCPM5-2B-5BitL-MLX

Base model

openbmb/MiniCPM5-2B

Quantized

(101)

this model

dalatexcoder
/

MiniCPM5-2B-5BitL-MLX

Model tree for dalatexcoder/MiniCPM5-2B-5BitL-MLX

Datasets used to train dalatexcoder/MiniCPM5-2B-5BitL-MLX