Autonomous Lunar Lander with DQN
A Deep Q-Network agent trained to land autonomously in LunarLander-v3.
- Year
- 2026
- Status
- Research project
- Focus
- Applied AI, Reinforcement learning
- Role
- Applied AI / Reinforcement Learning
Designed and evaluated a Deep Q-Network agent for autonomous landing in LunarLander-v3, covering problem framing, algorithm selection, PyTorch implementation, baseline comparison, multi-seed evaluation and ablation analysis.
LunarLander-v3 exposes an eight-dimensional continuous state space and four discrete actions. That combination makes tabular Q-learning impractical — there is no sensible way to discretise the state without either exploding the table or destroying the signal — which is what motivates a function approximator. The agent is a small fully connected network, 8 → 128 → 128 → 4 with ReLU activations, trained against a Bellman target with a replay buffer, a periodically synchronised target network and epsilon-greedy exploration.
A compact benchmark for learning decision-making from reward signals, useful when a control policy cannot be hand-coded reliably and the system has to improve through trial, feedback and evaluation.
What was built
- Justified deep Q-learning against classical tabular Q-learning for a continuous state space.
- Implemented the training loop in PyTorch with replay buffer, target network and epsilon-greedy exploration.
- Compared against random and heuristic baseline policies before training.
- Evaluated across five seeds, with every seed clearing the 200 average-reward threshold.
- Documented ablations, limitations and next steps including DDQN, prioritised replay and wind robustness.
At a glance
- Network
- 8 → 128 → 128 → 4
- Evaluation
- 5 seeds
- Success threshold
- 200 avg reward