LLM 本地化部署6：安装 Ollama 模型推理引擎

1. 安装

在线安装

        
        shell
    
curl -fsSL https://ollama.com/install.sh | sh

离线安装

        
        shell
    
curl -L https://ollama.com/download/ollama-linux-amd64.tgz -o ollama-linux-amd64.tgz
sudo tar -C /usr -xzf ollama-linux-amd64.tgz

权限设置

        
        shell
    
sudo useradd -r -s /bin/false -U -m -d /usr/share/ollama ollama
sudo usermod -a -G ollama $(whoami)

2. 配置

https://github.com/ollama/ollama/blob/main/docs/faq.md

        
        shell
    
# /etc/systemd/system/ollama.service.d/override.conf
sudo systemctl edit ollama

[Service]
Environment="OLLAMA_HOST=172.17.0.1:8888"
Environment="OLLAMA_KEEP_ALIVE=1h"
Environment="OLLAMA_MAX_LOADED_MODELS=2"
Environment="OLLAMA_NUM_PARALLEL=3"

默认位置： /usr/share/ollama/.ollama/models

**特别注意：**修改了默认主机后，运行ollama命令需要增加加前缀 OLLAMA_HOST=172.17.0.1:8888

例如：OLLAMA_HOST=172.17.0.1:8888 ollama ps

3. 自启动

sudo vim /etc/systemd/system/ollama.service

        
        
        
        
        shell
    
[Unit]
Description=Ollama Service
After=network-online.target

[Service]
ExecStart=/usr/bin/ollama serve
User=ollama
Group=ollama
Restart=always
RestartSec=3
Environment="PATH=$PATH"

[Install]
WantedBy=default.target

启动

        
        shell
    
sudo systemctl daemon-reload
sudo systemctl enable --now ollama

4. 卸载

        
        
        
        
        shell
    
sudo systemctl stop ollama
sudo systemctl disable ollama
sudo rm /etc/systemd/system/ollama.service
sudo rm $(which ollama)
sudo rm -r /usr/share/ollama
sudo userdel ollama
sudo groupdel ollama
sudo rm -rf /usr/local/lib/ollama

5. 导入GGUF模型

5.1. 创建 Modelfile 文件

vim DeepSeek-R1-Distill-Qwen-7B-GGUF/Modelfile

        
        
        
        
        shell
    
FROM DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf

TEMPLATE """{{- if .System }}{{ .System }}{{ end }}
{{- range $i, $_ := .Messages }}
{{- $last := eq (len (slice $.Messages $i)) 1}}
{{- if eq .Role "user" }}<｜User｜>{{ .Content }}
{{- else if eq .Role "assistant" }}<｜Assistant｜>{{ .Content }}{{- if not $last }}<｜end▁of▁sentence｜>{{- end }}
{{- end }}
{{- if and $last (ne .Role "assistant") }}<｜Assistant｜>{{- end }}
{{- end }}"""

PARAMETER stop "<｜begin▁of▁sentence｜>"
PARAMETER stop "<｜end▁of▁sentence｜>"
PARAMETER stop "<｜User｜>"
PARAMETER stop "<｜Assistant｜>"

5.2. 创建、运行模型

        
        shell
    
#ollama create <model_name> -f <path_to_Modelfile>
ollama create deepseek-r1-7b
ollama run deepseek-r1-7b
ollama list
ollama ps

目录