Local LLM (Local Large Language Model)

A local LLM is a large language model that runs on your own machine (laptop, server, or on-premises GPU) rather than through a cloud API. Tools like Ollama make it possible to run models like Phi-4, Llama 3.2, Mistral, and DeepSeek locally with GPU acceleration.

Why Local LLMs Matter for Security

• Data stays on your machine: prompts containing IAM policies, CloudTrail logs, Terraform templates, or customer data never leave your RAM
• No third-party processor: eliminates GDPR, HIPAA, and SOC 2 concerns about sending sensitive data to external APIs
• Offline capable: works in air-gapped environments and during incident response when network access may be restricted
• No per-token cost: run unlimited queries against your own hardware

Trade-offs

• Local models are smaller than cloud frontier models (8B-14B parameters vs 100B+)
• Quality is lower for complex reasoning tasks, but sufficient for policy review, code scanning, and log analysis
• Requires hardware with enough RAM (16-24GB minimum for useful models)

AWS Security Use Cases

• IAM policy analysis for privilege escalation risks
• CloudTrail log analysis for suspicious activity
• Terraform/CloudFormation security review
• Incident response assistance with NIST framework guidance
• Security code review for Lambda functions and application code