All Documents
3,528 documents available
Metrics
This document outlines the evaluation metrics available for assessing the performance of Retrieval Augmented Generation (RAG) systems, particularly focusing on the retrieval and generation components. The implementations can be found in `datapizza/evaluation/metrics.py`.
GenAI Benchmarks & Evaluation — Product-Based Companies
Understanding how to **benchmark, evaluate, and compare LLMs** is essential for roles at Google, OpenAI, Anthropic, Cohere, and AI research teams. This file covers the most important benchmarks, evaluation methodologies, and how to build custom evaluation harnesses.
After LangGraph node execution, convert messages
**RAGAS** (Retrieval-Augmented Generation Assessment) is a specialized evaluation framework designed to measure RAG pipeline performance through reference-free metrics, making it ideal for production systems. **LangGraph** is a state-based orchestration framework that structures AI workflows as directed graphs. Integrating these two creates a powerful system for building and evaluating complex RAG pipelines systematically.
IR-Copilot — Incident Response AI Assistant
Evaluation Scripts
The `create_test_set.py` script helps you interactively build a golden test dataset for evaluating the retrieval system.
Agents and LLMs
* how to measure hallucinations?
Day 20: Evaluation & Benchmarks 📏
root((Day 20: Evaluation & Benchmarks 📏))
Using Performance Metrics to Evaluate RAG Systems
title: "Data-Driven RAG Evaluation: Testing Qdrant Apps with Relari AI"
LLM Evaluation
LLM output evaluation — automated metrics, LLM-as-judge, A/B testing, regression testing. Use when measuring LLM output quality, comparing prompt or model versions, building an automated eval pipeline, setting up regression tests for prompt changes, or evaluating RAG systems and bias/safety.
📈 Trading RAG Mentor
> **Personal AI Trading Mentor** — A custom Retrieval-Augmented Generation (RAG) system built on momentum & price action video transcripts. Ask questions and get answers grounded exclusively in your own trading knowledge base.
AWS Certified Generative AI Developer – Professional (AIP-C01)
These are my personal study notes for the **AWS Certified Generative AI Developer – Professional (AIP-C01)** exam.
RAG System Testing Methodologies: A Comprehensive Guide
**Document Version:** 1.0
Domain 5: Testing, Validation, and Troubleshooting
**AIP-C01 Study Guide — Dr. Priya Ramanathan**
Understanding the Sources of Uncertainty - and Why Our Evals are Biased
Part 4 of *Iterating in the Dark:
AI Tester Interview Preparation Guide
1. [Core Competencies Overview](#core-competencies)
[BEE-30004] Evaluating and Testing LLM Applications
title: Evaluating and Testing LLM Applications
Agent Evaluation Reference Guide
Complete documentation for the `agent-eval` CLI, metrics, data formats, and customization.
Thesis Falsifier
A tool to aid researchers in assessing whether research papers adhere to scientific best practices. This application uses AI to automatically generate falsification forms, helping researchers verify the scientific robustness of their work across disciplines including social sciences and natural sciences.
QualRubric
The goal of a Qualifying Exam ("qual") is for a student to *effectively demonstrate that they have the knowledge and skills that will be needed to conduct meaningful research in their chosen subfield*. There are a number of key phrases in this sentence:
Glossary
*[Deutsche Version](GLOSSARY_DE.md)*
Instructions for Claude Code: n8n Meal Feedback LLM Evaluation Workflow
Create a plan to build an n8n workflow that evaluates multiple LLM prompts for generating meal feedback using a **thinking model to generate ground truth** for comparison.
! Project 3: Web APIs & NLP
In week four we've learned about a few different classifiers. In week five we learned about webscraping, APIs, and Natural Language Processing (NLP). This project will put those skills to the test.
Sprint 0 Marking Scheme
**Team Name:** BC Hub
Judging Rubric
**AI for Social Good Hackathon – SUST 2026**