Add ML installer-malware classifier

[forloopcodes]

Summary

Adds a machine-learning classifier that complements SafeInstall's hand-written static rules with a learned second opinion. A RandomForest weighs 57 behavioral features of an installer script together and returns a calibrated p(malicious); when it crosses the model threshold it becomes an ML Model finding that feeds the same energy-based risk score.

The model is trained offline and exported to pure Rust (src/ml_model.rs) via m2cgen, so inference at runtime needs no Python, no network, and no model file. Python and the exported Rust agree on p(malicious) to within 4.6e-11.

	benign installer	suspicious script

What's included

• src/features.rs — 57-feature extractor: the 10 static-rule hits as 0/1 features, plus entropy, line statistics, URL/IP counts, base64 density, 23 obfuscation and behavioral indicators (reverse shell, PowerShell cradles, credential access, anti-forensics, mining, exfiltration, …), and installer-kind one-hots. Counts are stored as ln(1+n). Exposed through a hidden safe features <file> --json subcommand so the training pipeline and the shipped binary compute an identical representation (enforced by tests).
• src/ml.rs + src/ml_model.rs — inference and the ML Model finding, wired into inspect/run right after analyzer::analyze() before recalculate(). ml_model.rs is generated by m2cgen.
• src/cli.rs / src/main.rs — --no-ml flag to skip the classifier; ML probability added to JSON output.
• src/source.rs — inspect local script files, not just URLs (needed for feature extraction and offline use).
• ml/ — reproducible pipeline (fetch_benign.py → build_dataset.py → train.py), a notebook (train.ipynb), figures (confusion matrix, ROC, SHAP), and screenshots. data/features.csv and metrics are committed; the raw script corpus is git-ignored.

Results (held-out test set, threshold 0.53)

Metric	Value
Precision	0.992
Recall	0.992
F1	0.992
ROC AUC	0.999
Python ↔ Rust parity	4.6e-11

RandomForest and GradientBoosting were compared; GB scored marginally higher (F1 0.985 vs 0.974 at threshold 0.5) but only RandomForest exports to dependency-free Rust via m2cgen, so it is the deployed model.

Data & safety

The benign class is features of real installer scripts (rustup, nvm, bun, Deno, Homebrew, Docker, Helm, pyenv, Starship, poetry, Tailscale, and dozens more, plus local system scripts). The malicious class is synthesized in feature space from documented MITRE ATT&CK indicator distributions, layered onto structural features borrowed from real benign scripts. No malware is stored, committed, or executed anywhere in this repository. The synthetic positive class is a stated limitation (metrics measure separability of the modeled attack distribution); swapping in a vetted real-malware feature corpus is the primary future-work item and changes nothing about the feature contract or the Rust export. Full methodology in ml/README.md.

Testing

• cargo test — 37 tests pass (feature extraction, inference thresholds, the Python↔Rust feature-order contract, benign-not-flagged / dense-indicator-flagged integration cases).
• cargo build --release — clean, no warnings.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Lb5wpChn4hHTRMNcRmBNpB

---

Generated by Claude Code