Non Functional Testing For Llm, Chatbots And Ai Models

What you'll learn
Understand how AI is working
Understand basic software testing
Understand how AI is tested compared to traditional software
Gain knowledge on testing for ethics
Demo on testing Chat GPT with automated Tools
Understand Adversial Testing techniques
Understand how to test for a human like conversation
Create a framework for testing bias, toxicity and hate with PerspectiveAPI
Requirements
basic experience with software testing
basic coding experience ( but not needed)
optional - GPT model 4 subscription (but not needed)
desire to learn the hottest skill on the market
desire to learn the hottest skill on the market
Description
Welcome to "Non Functional Testing for LLM, Chatbots and AI Models" your comprehensive guide to mastering the fundamentals of testing AI systems. Whether you're a developer, data scientist, or AI enthusiast, this course will provide you with the knowledge and skills needed to assess, improve, and ensure the reliability, performance, safety, and ethical integrity of AI technologies.What You Will Learn:Introduction to AI Testing: Understand the critical importance of testing AI systems, addressing both technical performance and ethical considerations. Learn about the potential impacts of AI failures and how responsible testing mitigates these risks.Special Focus on Foundation Models and LLMs: Dive deep into the unique challenges of testing large language models and foundational AI systems, which are driving innovation across multiple industries.AI System Evaluations: Learn how to design and implement effective testing frameworks for AI-based systems, utilizing both manual and automated tools to improve system performance and safety.Adversarial AI Testing: Understand how to evaluate the robustness of AI models through adversarial testing techniques, assessing how well AI systems resist manipulation and errors when exposed to malicious inputs.PerspectiveAPI for Ethical and Toxicity Testing: Learn how to integrate the PerspectiveAPI and other tools to test AI systems for ethical compliance and detect harmful or toxic outputs, ensuring AI systems uphold safety and ethical standards.Humanness in AI: Explore the concept of evaluating the "humanness" of AI responses. Learn how to test whether AI systems generate outputs that are human-like, contextually aware, and empathetic in their interactions.Ethical AI: Delve into the risks associated with AI and the ethical dimensions of AI development. Learn how to test AI systems for bias, fairness, and transparency, ensuring adherence to responsible AI practices.Testing ChatGPT and Chatbots Using APIs in MLOps: Learn to test and evaluate conversational models like ChatGPT through APIs, and understand how to integrate these tests into MLOps pipelines for continuous AI improvement.Case Studies: Review real-world examples of AI testing, learning from common pitfalls and best practices used in the field to ensure AI reliability and safety.Who This Course Is For:This course is designed for individuals seeking a comprehensive understanding of the techniques and practices required for testing AI systems. Whether you are starting a career in AI, enhancing your professional skills, or interested in the technical and ethical mechanisms behind AI system reliability, this course offers valuable insights.Enroll now to start mastering the critical skill of testing AI systems, ensuring that you are equipped to contribute to the development of safe, reliable, and ethically sound AI technologies!
Overview
Section 1: Introduction
Lecture 1 Introduction
Lecture 2 About your instructor
Lecture 3 A word of introduction on Generative AI
Lecture 4 History of AI
Lecture 5 What you will learn in this material
Section 2: Setup Environment
Lecture 6 Install VS Code
Lecture 7 Install NodeJS and NPM
Lecture 8 Install Python
Lecture 9 Install Python Dependencies - PIP
Section 3: Introduction to Artificial Intelligence
Lecture 10 What makes up AI
Lecture 11 Where do LLMs fit into AI
Lecture 12 Introduction to Natural Language Processing
Lecture 13 Introduction to Machine Learning (ML)
Lecture 14 Machine Learning - Supervised ML
Lecture 15 Machine Learning - Unsupervised ML
Lecture 16 Machine Learning - Reinforced ML
Lecture 17 Neural Networks and Deep Learning
Lecture 18 Importance of Training Data
Lecture 19 What actually is GEN AI
Section 4: Introduction to LLM Basic Testing
Lecture 20 Types of Testing in Software
Lecture 21 Testing Types for LLMs | Foundation Models
Section 5: Automated Testing Framework with Postman and ChatGPT
Lecture 22 What is a token in LLMs
Lecture 23 Chat GPT-API - Create Subscription
Lecture 24 Get an OPENAI API Key
Lecture 25 Installing Postman and first API Test
Lecture 26 API Collections and making Results Deterministic
Lecture 27 Installing Newman and Running with the CLI
Lecture 28 Demo - GitHub - Adding Tests in ML OPS Pipeline
Section 6: Toxicity Testing Framework for LLMs
Lecture 29 Perspective Service - Bias Detection Service
Lecture 30 Get a Perspective API Key
Lecture 31 Demo - VS Code - Call Perspective API
Lecture 32 Demo - Python - Test AI Response against Perspective APIs
Section 7: Adversial/Security Testing for LLMs
Lecture 33 Adversial attacks for LLMS and Red Team
Lecture 34 Prompt Injection Attack
Lecture 35 FUZZ Testing
Lecture 36 Denial of Service Attacks
Lecture 37 Adversial Attack Examples
Lecture 38 Poisoning attack
Lecture 39 Privacy Leakage Testing
Lecture 40 Evasion Attacks
Section 8: Non - Functional Testing - Human in AI
Lecture 41 What is non functional Testing for LLMs
Lecture 42 Disclaimer on Non Functional Testing
Lecture 43 Non functional Testing AI Models | LLMs - Ethical Alignment
Lecture 44 Non functional Testing AI Models | LLMs - Explainability
Lecture 45 Non functional Testing AI Models | LLMs - User Interaction Robustness
Lecture 46 Non functional Testing AI Models | LLMs - Context Preservation
Lecture 47 Non functional Testing AI Models | LLMs - Creativity and Novelty
Section 9: Ethical Consideration for AI
Lecture 48 Asimov's 3 Laws of Robots
Lecture 49 DEMO - Why we need ethical and responsible AI Systems
Lecture 50 AI and Biases
Lecture 51 GEN AI and Privacy
Lecture 52 GEN AI and Intellectual Property
Lecture 53 Gen AI and Deep Fake
Lecture 54 Hallucinations
Lecture 55 OPENAI-CHAT GPT Moderation Service
Lecture 56 Google Moderation Service
Lecture 57 Spot a Fake - Demo Chat GPT Watermark on Dall E
Citizen Developer,Software testers,Quality engineers,Social Engineers,Prompt Engineers,Product Managers,Engineering Directors

Homepage

https://www.udemy.com/course/non-functional-testing-for-llm-chatbots-and-ai-models/