Unity + ML-Agents Toolkit AI Full Syllabus

Module 1: Introduction to ML in Gaming

• Role of AI in modern games
• Rule-based vs Learning-based Game AI
• What is Unity ML-Agents Toolkit?
• Use cases: NPCs, opponents, bots, and dynamic difficulty adjustment
• Prerequisites: Unity basics, Python, basic ML knowledge

Module 2: Getting Started

• Installing Unity (Unity Hub & LTS version)
• Setting up a new Unity 3D project
• Installing ML-Agents Toolkit via GitHub or Unity Package Manager
• Installing Python, ML-Agents Python API, dependencies (Anaconda / pip)
• Testing ML-Agents with demo scenes

Module 3: ML-Agents Toolkit Architecture

• Core Components:
  • Agents
  • Environments
  • Brains (deprecated → now handled by Python policies)
  • Sensors
  • Heuristics
• Training pipeline overview: Unity ↔ Python communication

Module 4: Creating Custom Environments

• Designing a simple game world (e.g. maze, ball collector, platformer)
• Adding ML-Agent scripts to GameObjects
• Creating Agent scripts (C#)
• Defining observations and actions
• Reward signal design
• Training and inference-ready environment setup

Module 5: Observations, Actions, and Rewards

• Vector Observations vs Visual Observations (camera-based)
• Discrete vs Continuous Actions
• Action Masks
• Reward shaping best practices
• Balancing sparse vs dense rewards
• Designing curriculum learning

Module 6: Training the Agent (Python Side)

• Using mlagents-learn CLI tool
• Configuring config.yaml for training (hyperparameters, PPO/SAC)
• Running training: Unity ↔ Python via gRPC
• Monitoring training with TensorBoard
• Saving and exporting trained models

Module 7: Algorithms in ML-Agents

• Proximal Policy Optimization (PPO)
• Soft Actor-Critic (SAC)
• Imitation Learning (BC)
• Self-play
• Curriculum Learning
• Hyperparameter tuning strategies

Module 8: Running Inference in Unity

• Loading trained .nn models back into Unity
• Using Inference mode with Behavior Parameters
• Switching between Heuristic, Inference & Training modes
• Behavior Trees vs ML Agents: Integration options
• Deploying ML-controlled characters in real games

Module 9: Advanced Concepts

• Multi-agent environments
• Communication between agents
• Memory (Recurrent neural networks, RNN support)
• Domain Randomization
• Reward hacking detection & prevention
• Debugging and logging in ML-Agents

Module 10: Visual Observations & Camera-based Learning

• Setting up agent camera views
• Processing visual observations with CNNs
• Optimizing performance with smaller image resolutions
• Using raycasts and sensors (Ray Perception Sensor Component)

Module 11: Imitation & Human Demonstration Learning

• Collecting demonstrations using Recorder
• Behavior Cloning (BC) setup
• Comparing RL vs BC performance
• Use case: teaching bots human-like behavior

Module 12: Real Game Scenarios

• Training AI for:
  • Platformers (jumping bots)
  • Shooters (aim, dodge, patrol)
  • Racing agents
  • Collect-and-avoid logic games
• Dynamic difficulty agents
• RL-based procedural content (level auto-adjustment)

Module 13: Saving, Sharing & Deployment

• Exporting .nn models
• Deploying to WebGL, Android, or PC builds
• Packaging environments for research or games
• Creating public demos or portfolio showcases