Practical Jupyter notebooks from Andrew Ng and Giskard team's "Red Teaming LLM Applications" course on DeepLearning.AI.
-
Updated
Apr 8, 2024 - Jupyter Notebook
Practical Jupyter notebooks from Andrew Ng and Giskard team's "Red Teaming LLM Applications" course on DeepLearning.AI.
✍️ Collaborate on writing technical content for the Giskard Community
Adversarial LLM red-teaming with Giskard: automated vulnerability scans of DeepSeek-R1, GPT-4o-mini & Llama 3.2 (prompt injection, hallucination, harmful output) plus tabular ML scanning, with per-model HTML reports
Mutant is a red team and behavioral data generation library for LLMs, RAG pipelines, and AI Agents. It provides two powerful capabilities
Develop a system to generate diverse types of subjective questions from PDFs and automate the grading of answers.
Independent SR 11-7-style model validation of an ML trading strategy - deepchecks + giskard, formal report, reproducible. Verdict: not approved for deployment.
LLM evaluation & AI safety suite for a live chatbot — custom LLM-as-a-Judge metrics, jailbreak/prompt-injection red teaming, RAG groundedness (Ragas, Giskard), Langfuse tracing, plus Playwright E2E and k6, gated nightly in CI.
RAG experiments with LangChain: PDF Q&A over OpenAI and Perplexity models, evaluated with Giskard
Multi-agent LLM-driven SOC pipeline (n8n + Ollama), adversarially red-teamed against the CSA Agentic AI Red Teaming Guide.
Different approaches to evaluate RAG !!!
Security testing toolkit for AI agents
Red-team assessment of a financial-services chatbot and an e-commerce agent: manual probing, Giskard scanning, and adversarial simulation. 16 findings mapped to the OWASP Top 10 for LLMs, with an audit trail.
The complete pipeline for a fine-tuning of a classifier model.
PRML pre-registration for Giskard scenario results: commit an eval claim to a SHA-256 before the run, then verify the result against it.
To associate your repository with the giskard topic, visit your repo's landing page and select "manage topics."