Run LLMs Locally (Harbor)

Trust Score 74/100

from harbor2,955

Deploy, configure, and troubleshoot a full local LLM stack (Ollama, llama.cpp, vLLM, Open WebUI, SearXNG, Open Terminal) using the Harbor toolkit.

triggers:run llmharbor uppull modelgpu troubleshootingopen webuivllmllama.cpp

GitHub SKILL.md

What it does

Harbor's Run LLMs skill documents how to install, configure, and operate a full local LLM stack. It covers bringing up default services, pulling models from Ollama and HuggingFace, managing backends (Ollama, llama.cpp, vLLM), enabling web search (SearXNG), and adding code-execution via Open Terminal. The skill includes practical troubleshooting playbooks for GPU issues, model OOMs, service restarts, and network/tunnel exposure.

When to use it

Use this skill when you want a reproducible, containerized local AI environment for experimentation or small-scale production: spinning up an LLM UI, switching backends, pulling models, diagnosing GPU and startup failures, or enabling web-augmented chat and code execution. It is intended for devs running on local machines, homelabs, or servers.

What's included

Scripts: none bundled in the skill directory, but Harbor repo contains CLI tooling (harbor) referenced throughout.
References: detailed command examples and configuration tips are embedded in the skill body (pull, restart, logs, env overrides).
Instructions: step-by-step workflows for initial setup, model pulls, backend switching, VRAM optimization, and common troubleshooting playbooks.

Compatible agents

Agents and CLIs that run terminal commands or manage containers will find this useful (GitHub CLI, shell-based agents, automation runners, and agent tooling that can execute Docker/CLI operations).

Audit Summary

Comprehensive guide skill for setting up and running local LLMs using Harbor (a containerized LLM toolkit). Covers Ollama, llama.cpp, vLLM, Open WebUI, SearXNG, and Open Terminal with detailed decision trees, CLI references, configuration, and troubleshooting. No bundled scripts — purely instructional. The only security concern is the curl-pipe-bash install method.

Watch Out

Install command uses curl | bash pattern (curl https://av.codes/get-harbor.sh | bash) — users should inspect the script first
harbor logs command tails indefinitely and will hang agent executions — must use docker logs instead
Default JWT secret 'h@rb0r' should be changed in production

Notes

Very well-written reference skill. Decision trees are excellent for autonomous agent use. The curl | bash install is standard for CLI tools but a security deduction applies. No scripts to test. SKILL.md is very long (~800 lines) which could benefit from splitting into references/ but remains well-organized with clear sections.

Information

Repository: harbor
Stars: 2,955

Trust Score

Overall74

Security65

Code Quality92

Architecture68

Usefulness82

Related Skills

Yielding Bear — Unified LLM Routing

A unified LLM routing API that automatically routes requests to the cheapest capable model across providers to reduce inference cost by ~60-80%.

Sync Production Database to Development

Run a safe, scripted workflow to download production database dumps and restore them into a development environment (download-only and restore options).

Markdrop

Convert PDFs to structured Markdown or interactive HTML and generate AI-powered descriptions for images and tables using multiple LLM providers.

Terragrunt Infrastructure

Best-practice patterns and operational guidance for Terragrunt/OpenTofu infrastructure: catalog/live/module layout, state backend setup, and deployment workflow

Azure Impact Reporting — Integration & Troubleshooting

Guidance for configuring, troubleshooting, and integrating Azure Impact Reporting with Monitor, Logic Apps, Service Health and HPC guest health APIs.

Microsoft Foundry Classic — Expert Guidance

Comprehensive guidance for building, configuring, troubleshooting, and deploying Microsoft Foundry Classic agents and integrations.

Docker Compose Generator

Generate and validate Docker Compose files for multi-container apps with templates, best-practice guidance, and deployment/runbook outputs.

Code Audit

Perform professional code security audits across 9 languages with configurable quick/standard/deep modes and Docker-backed verification.

Back to Skills