SVM-Based LLM Text Detector Achieves 85% Accuracy
Original: Detecting LLM-Generated Texts with “Classical” Machine Learning
Why This Matters
Demonstrates that classical ML can rival LLM-based detectors for AIGC identification at low inference cost.
Developer lyc8503 built an LLM-generated text detector using scikit-learn SVM, achieving approximately 85% single-sentence accuracy on test data. The tool targets AI-generated web fiction on platforms like Lofter, rejecting perplexity-based methods in favor of classical ML statistical patterns.
Developer lyc8503 published a blog post (dated March 1, 2026) detailing the construction of an AI-generated content (AIGC) detector using classical machine learning—specifically a Support Vector Machine (SVM) via scikit-learn—rather than modern LLM-based perplexity scoring.
The project was motivated by the author's frustration with AI-generated fanfiction flooding the Lofter platform. After an earlier failed attempt using text perplexity (measuring token probability rankings via an LLM), which yielded too many false positives and negatives with no reliable threshold, the author pivoted to a statistical pattern approach.
The final pipeline covers: data generation, SVM training, a JavaScript implementation for a browser-based demo, and adversarial testing. The demo is publicly accessible at lyc8503.github.io/AITextDetector, with core code and model files on GitHub (lyc8503/AITextDetector).
The model was not trained on general-purpose data and has not undergone rigorous optimization; single-sentence detection accuracy stands at roughly 85% on the test set. The post also covers attack and defense scenarios, including a classic translation bypass method and LLM prompt-based evasion attempts. The author notes that as of early 2026, mainstream LLM outputs exhibit strong enough statistical regularities to be distinguishable from human writing using traditional ML classifiers, suggesting this may underlie many commercial AIGC detection services.