Neura Market
model-audit logo

model-audit

Free

LLM 蒸馏检测与模型指纹审计 — 文本溯源、身份验证、蒸馏关系判定，守护模型知识产权。

FreeFree tier
Inputs: textOutputs: text
Type
Open Source
Visit Website GitHub (40 stars)

About model-audit

ModelAudit is an open-source framework for detecting unauthorized LLM knowledge distillation using statistical forensics. It extracts model fingerprints through 20 behavioral probes covering 10 dimensions (self-cognition, safety boundaries, injection tests, reasoning, creativity, multilingual, format control, role-play, code generation, summarization), then applies hypothesis testing to determine if a student model has copied a teacher model. The framework integrates four complementary methods: LLMmap (behavioral probing), DLI (lineage inference via Jensen-Shannon divergence), REEF (CKA representation similarity), and StyleAnalysis (12-family stylistic signatures). It achieves 100% detection accuracy on a built-in benchmark of 6 model families (14 samples). ModelAudit works in black-box scenarios (API-only models), provides a CLI and MCP server, and is designed specifically for model IP protection—not general model evaluation.

Key Features

Four complementary detection methods: LLMmap (behavioral probing), DLI (lineage inference), REEF (CKA similarity), StyleAnalysis (stylistic signatures)
20 behavioral probes across 10 dimensions for model fingerprint extraction
Statistical hypothesis testing framework (H0: M_S ⟂ M_T vs H1: M_S ← M_T) with configurable threshold
Built-in benchmark achieving 100% detection accuracy on 6 model families (14 samples)
Works in black-box scenarios (requires only input-output access)
CLI and MCP server support for integration into workflows
Addresses three core challenges: distillation opacity, stylistic convergence, cross-model incomparability

Pros & Cons

Pros
  • Achieves 100% detection accuracy on the built-in benchmark
  • Combines multiple detection methods for robust results
  • Works with black-box models (API-only access) through behavioral probing
  • Open-source and free to use
  • Provides CLI and MCP server for flexible deployment
Cons
  • Focused solely on detecting distillation, not a general-purpose model evaluation tool
  • Requires access to model outputs for all 20 behavioral probes
  • Detection performance on unseen model families may vary (benchmark covers 6 families)

Best For

Detecting unauthorized knowledge distillation of LLMsModel intellectual property protection and auditVerifying model lineage and cross-model relationshipsForensic analysis of model behavior to identify copying

FAQ

What is ModelAudit?
ModelAudit is an open-source framework that detects unauthorized LLM knowledge distillation using statistical forensics, behavioral probing, and representation similarity analysis.
How does ModelAudit detect distillation?
It extracts model fingerprints from 20 behavioral probes covering 10 dimensions, then applies hypothesis testing (Pearson correlation) to determine if one model is likely distilled from another. A score above the default threshold of 0.7 indicates suspicion.
What methods does ModelAudit use?
Four complementary methods: LLMmap (behavioral probing), DLI (lineage inference via Jensen-Shannon divergence), REEF (CKA representation similarity), and StyleAnalysis (12-family stylistic signatures).
Can ModelAudit work with black-box models?
Yes. It requires only input-output access to the model (API calls), making it suitable for black-box scenarios where weights are not available.