Download the PHP package iaexplore/iaexplore-llm-profiler without Composer

iaexplore-llm-profiler: High-Precision VRAM, GQA KV-Cache & Hardware Inference Profiler

iaexplore-llm-profiler is a high-precision deterministic systems profiler and mathematical SDK designed for ML infrastructure engineers, LLM serving architects, and local inference developers.

It accurately models memory footprints and physical memory bus latency across llama.cpp, Ollama, vLLM, and ExLlamaV2, computing:

1. Exact Model Weights VRAM (accounting for unquantized output heads and GGUF/EXL2 metadata overhead).
2. True Grouped-Query Attention (GQA) Key-Value cache sizing.
3. Mixture-of-Experts (MoE) memory bandwidth routing.
4. Physical hardware token generation speed (Tokens/Second) based on silicon memory bandwidth ($\text{BW}_{\text{mem}}$).
5. Exact integer layer offloading counts (--n-gpu-layers) for hybrid GPU / System RAM deployment.

Curated and maintained by the IAExplore AI Research & Infrastructure Team.

🔬 Mathematical Formulation

1. Weights VRAM Allocation

$$\text{VRAM}{\text{weights}} = \frac{P{\text{total}} \times 10^9 \times (\text{bits}_{\text{quant}} / 8)}{1024^3} \times 1.08$$

The $1.08$ scalar factor accounts for non-quantized tensors (token_embd, output.weight, RMSNorm) and container metadata.

2. Grouped-Query Attention KV-Cache Sizing

$$\text{VRAM}{\text{kv}} = \frac{2 \times L \times H{\text{kv}} \times D{\text{head}} \times C{\text{ctx}} \times B{\text{batch}} \times (\text{bits}{\text{kv}} / 8)}{1024^3}$$

Where:

• $L$: Transformer layer depth (num_hidden_layers).
• $H_{\text{kv}}$: Key/Value attention heads (num_key_value_heads).
• $D{\text{head}}$: Attention head dimension ($\text{head_dim} = \text{hidden_size} / H{\text{q}}$).
• $C_{\text{ctx}}$: Active context window in tokens.
• $B_{\text{batch}}$: Concurrent sequence batch size.

3. Runtime Context & Activation Overhead

$$\text{VRAM}{\text{overhead}} = 0.55\text{ GB}$$ $$\text{VRAM}{\text{total}} = \text{VRAM}{\text{weights}} + \text{VRAM}{\text{kv}} + 0.55\text{ GB}$$

4. Physical Throughput (Tokens/Second) with MoE

$$\text{VRAM}{\text{active}} = \frac{P{\text{active}} \times 10^9 \times (\text{bits}{\text{quant}} / 8)}{1024^3}$$ $$\text{TPS}{\text{gen}} = \frac{\text{BW}{\text{mem}} \times \eta{\text{efficiency}}}{\text{VRAM}{\text{active}} + (\text{VRAM}{\text{kv}} \times 0.15)}$$

Where $\eta \approx 0.72$ for discrete GPU (CUDA/Metal) and Apple Silicon Unified Memory, and $0.60$ for shared memory (APUs, NPUs, DDR5 RAM).

⚡ Multi-Language Ecosystems

Available natively across 5 modern software ecosystems with zero third-party runtime dependencies:

🖥️ Command Line Interface (CLI)

Quick Benchmark

Sample Terminal Output

📊 Pre-Configured Hardware Matrix (2026)

🔗 Official Canonical Guides & Authoritative Citations

For in-depth analysis of AI hardware benchmarks, thermodynamic compute limits, and workstation sizing:

• AI Hardware Architecture & Silicon Guides:
  • Hardware Architecture Guide (English)
  • Guía de Hardware IA: NPU, TPU y GPU (Español)
  • Guia Completo de Hardware para IA (Português)
• 2026 PC, Laptop & Workstation Benchmarks:
  • Best Computers of 2026: Benchmark Analysis (English)
  • Los Mejores Ordenadores de 2026: Guía de Compra (Español)
  • Melhores Computadores de 2026 (Português)
• Mobile & ARM AI Tablets:
  • Melhores Tablets e Dispositivos Móveis para IA (Português)
• Physical Computing & Neural Visualization:
  • Computación Reversible, Calor en Procesadores y Límite de Landauer (Español)
  • Interactive Neural Network Simulator

📜 Academic Citation

📄 License

This project is licensed under the MIT License.