1× knowledge

Generative AI / Subject

Advanced

Engineer production generative AI with evaluation, LLMOps, serving, observability, inference optimization, security, red teaming, responsible AI and governance.

Your Progress

Learning Summary

0 of 101 topics completed
Quizzes Completed 0Best Quiz Score —Current Level Not startedContinue Choose a topic

Suggested sequence

Learning roadmap

Topics follow the Admin display order

Advanced

101 topics
  1. LLM Evaluation Fundamentals
  2. Benchmarking
  3. Accuracy Evaluation
  4. Relevance Evaluation
  5. Faithfulness Evaluation
  6. Groundedness Evaluation
  7. Hallucination Evaluation
  8. Toxicity Evaluation
  9. Bias Evaluation
  10. Safety Evaluation
  11. Retrieval Evaluation
  12. Context Precision
  13. Context Recall
  14. Answer Relevance
  15. Human Evaluation
  16. LLM-as-a-Judge
  17. Pairwise Evaluation
  18. Regression Testing
  19. Golden Datasets
  20. Evaluation Pipelines
  21. LLMOps Fundamentals
  22. Model Deployment
  23. Model Serving
  24. API Deployment
  25. Model Monitoring
  26. Prompt Monitoring
  27. Token Monitoring
  28. Cost Monitoring
  29. Latency Monitoring
  30. Quality Monitoring
  31. Observability
  32. Tracing
  33. Logging
  34. Evaluation in Production
  35. A/B Testing
  36. Canary Deployment
  37. Model Versioning
  38. Prompt Versioning
  39. Dataset Versioning
  40. Model Registry
  41. CI/CD for AI
  42. AI Gateway
  43. Load Balancing
  44. Autoscaling
  45. Quantization
  46. 8-bit Quantization
  47. 4-bit Quantization
  48. GPTQ
  49. AWQ
  50. GGUF
  51. Model Pruning
  52. Knowledge Distillation
  53. Speculative Decoding
  54. Continuous Batching
  55. Prefix Caching
  56. KV Cache Optimization
  57. Flash Attention
  58. Tensor Parallel Inference
  59. GPU Inference
  60. CPU Inference
  61. Edge Inference
  62. AI Security Fundamentals
  63. Prompt Injection
  64. Indirect Prompt Injection
  65. Jailbreaking
  66. Data Leakage
  67. Training Data Leakage
  68. Model Extraction
  69. Model Inversion
  70. Membership Inference
  71. Adversarial Inputs
  72. Poisoning Attacks
  73. RAG Poisoning
  74. Tool Abuse
  75. Agent Security
  76. Excessive Agency
  77. Insecure Output Handling
  78. Secret Management
  79. Sandboxing
  80. Access Control
  81. AI Red Teaming
  82. Guardrails
  83. Responsible AI
  84. AI Ethics
  85. AI Bias
  86. Fairness
  87. Transparency
  88. Explainability
  89. Privacy
  90. Copyright
  91. Intellectual Property
  92. Data Governance
  93. AI Governance
  94. Model Cards
  95. Dataset Documentation
  96. Content Provenance
  97. AI-Generated Content Detection
  98. Watermarking
  99. Deepfake Detection
  100. Human Oversight
  101. Risk Management

Topics

101 published topics

  1. 01 LLM Evaluation FundamentalsLearn LLM Evaluation Fundamentals through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  2. 02 BenchmarkingLearn Benchmarking through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  3. 03 Accuracy EvaluationLearn Accuracy Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  4. 04 Relevance EvaluationLearn Relevance Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  5. 05 Faithfulness EvaluationLearn Faithfulness Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  6. 06 Groundedness EvaluationLearn Groundedness Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  7. 07 Hallucination EvaluationLearn Hallucination Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  8. 08 Toxicity EvaluationLearn Toxicity Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  9. 09 Bias EvaluationLearn Bias Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  10. 10 Safety EvaluationLearn Safety Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  11. 11 Retrieval EvaluationLearn Retrieval Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  12. 12 Context PrecisionLearn Context Precision through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  13. 13 Context RecallLearn Context Recall through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  14. 14 Answer RelevanceLearn Answer Relevance through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  15. 15 Human EvaluationLearn Human Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  16. 16 LLM-as-a-JudgeLearn LLM-as-a-Judge through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  17. 17 Pairwise EvaluationLearn Pairwise Evaluation through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  18. 18 Regression TestingLearn Regression Testing through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  19. 19 Golden DatasetsLearn Golden Datasets through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning
  20. 20 Evaluation PipelinesLearn Evaluation Pipelines through a detailed Generative AI lesson with a relevant example, verification, exercises, safety analysis and practice questions.Advanced · 14 min read · 180 min learning