All projects
GarudaAI
Local LLM Setup and Remote Access CLI
Running a local LLM requires knowing your GPU specs, picking quantization levels, and configuring remote access. GarudaAI collapses the entire setup into one command.
What it does
Linux CLI tool that profiles GPU specs, determines which local LLMs can run efficiently on the hardware, installs Ollama with the optimal model, and exposes a remote-access endpoint reachable from any phone on the network.
What I built
- GPU profiling pipeline that matches VRAM and compute against model quantization requirements to recommend the best-fit local LLM.
- One-command setup: installs Ollama, pulls the right model, and tunnels a mobile-accessible endpoint for on-the-go inference.
How it works
- 01Python CLI
- 02GPU Hardware Profiler (VRAM + compute detection)
- 03Ollama Auto-Installer
- 04Model Pull
- 05Network Tunnel (mobile remote access)
Features
- Profiles GPU VRAM and compute against quantization tables (Q4_K_M, Q5, Q8, fp16)
- Recommends the highest-quality model your hardware can actually run
- Automatically installs Ollama and pulls the matched model without manual steps
- Exposes a mobile-accessible inference endpoint via network tunnel
- Supports LLaMA 3, Mistral, Phi-3, and all Ollama-compatible model families