Closed-Loop Detection Validation Engine

Pillar 7: TP/FP/F1 Pipeline

Automated regression pipeline testing detection rules against 500 Curated Malicious Binaries + 10,000 Clean Windows System32 Binaries to eliminate False Positives.

Active Detection Rule:

Validation GradeINITIAL DRAFT

D (UNSAFE)

GRADE D / F (HIGH FALSE POSITIVE RISK)

False Positives (FP)

184/ 10,000 clean

⚠️ Collides with Windows System Files

True Positives & Recall

498/ 500 malware (99.6%)

Missed Malware (FN): 2 samples

F1 Harmonic Score

84.3%Precision: 73%

Latency: 2.14ms per 1k files

Statistical Confusion Matrix (10,500 Total Binaries)

Target: WannaCry Ransomware Worm (MS17-010)

True Positives (TP)498Malicious binaries correctly flagged (99.6% Recall)

False Positives (FP)184Clean Windows files incorrectly alerted

False Negatives (FN)2Malicious samples that bypassed the rule

True Negatives (TN)9,816Clean binaries safely passed without alert

Test Dataset Corpus & Performance Latency

🦠 Malicious Test Corpus (500 Samples)500 PEs / 1.2 GB

Comprises WannaCry, LockBit 3.0, Cobalt Strike Beacons, SikoMode, RedLine Stealer, Emotet, and Qakbot packed variants.

🛡️ Benign Windows Corpus (10,000 Binaries)10,000 PEs / 42.8 GB

Curated pristine Windows 10/11 Enterprise x64/x86 system binaries, Sysinternals suite, Visual Studio toolchain, and top 200 legitimate enterprise vendor executables.

⚡ Engine Detection Speed2.14 ms / 1,000 files

Throughput: 4,672 files/sec scanned in multi-threaded YARA-X memory stream.