Neura Market
giskard-oss logo

giskard-oss

Free

🐢 Open-Source Evaluation & Testing library for LLM Agents

CybersecurityFreeFree tier
Type
Open Source
Company
Giskard
Visit Website GitHub (5.3k stars)

About giskard-oss

Giskard is an open-source Python library for LLM testing and evaluation. It provides automated detection of security vulnerabilities via LLM Scan and business logic failures via the RAG Evaluation Toolkit. The library is available on GitHub and forms the basis for a course on Red Teaming LLM Applications on Deeplearning.AI. It offers a programmatic interface for technical users with basic testset generation and testing capabilities.

Key Features

Automated detection of security vulnerabilities using LLM Scan
Automated detection of business logic failures using RAG Evaluation Toolkit
Programmatic interface for technical users (Python SDK)
Basic testset generation for business and security failures
Integration with GitHub and used in Deeplearning.AI red teaming course

Pros & Cons

Pros
  • Free and open-source
  • Automated vulnerability and business logic scanning
  • Backed by research and collaborations with leading AI organizations
  • Complements enterprise Giskard Hub with more advanced features
Cons
  • Limited to basic testing capabilities compared to Giskard Hub
  • Requires programming knowledge (Python)
  • Not designed for non-technical users

Best For

LLM application testingRed teaming and security auditingBusiness logic validation for RAG systemsEducational use for learning LLM evaluation

Alternatives to giskard-oss

h4cker

This repository is maintained by Omar Santos (@santosomar) and includes thousands of resources related to ethical hacking, bug bounties, digital forensics and incident response (DFIR), AI security, vu

ai-captcha-bypass

AI Captcha Bypass

backdoor-learning-resources

A list of backdoor learning resources

claude-bug-bounty

AI-powered bug bounty hunting from your terminal - recon, 20 vuln classes, autonomous hunting, and report generation. All inside Claude Code.

agentic-radar

A security scanner for your LLM agentic workflows

FAQ

What is Giskard Open Source?
Giskard Open Source is a free, open-source Python library for LLM testing and evaluation, providing automated detection of security vulnerabilities and business logic failures.
How does Giskard Open Source differ from Giskard Hub?
Giskard Hub is an enterprise platform with advanced team collaboration, continuous red teaming, and a UI for business users, while the Open Source version offers basic testing capabilities via a programmatic SDK.
Is Giskard Open Source free to use?
Yes, it is open-source and available for free on GitHub.