Aplicación de aprendizaje por refuerzo en un entorno de videojuego desarrollado para la simulación de estrategias de speedrun con un agente inteligente

Abstract

This research develops a system based on Reinforcement Learning (RL) for the optimization of strategies in a 2D platform speedrun videogame using Godot Engine 4.5.1 and Python 3.10. The objective of the study is to implement an autonomous support agent within the game, capable of minimizing level completion time while responding effectively to different in-game scenarios through the application of RL architectures. The simulation environment is implemented using Godot Engine 4.5.1, incorporating customized physics and a progressive level design that supports the agent’s incremental learning process. The system follows a client–server architecture, in which the game engine acts as the client and communicates in real time with an RL agent implemented in Python via TCP/IP sockets. This communication enables the reception, execution, and storage of the agent’s optimal policies using the Proximal Policy Optimization (PPO) algorithm within the Stable Baselines3 framework. Experimental results report an average client–server communication latency of 1.3 ms, along with a dense reward function specifically designed to encourage efficient forward movement, strategic use of the dash mechanic, and the reduction of redundant actions through penalization criteria. During the initial 10,000 training steps, the agent exhibits exploratory learning behavior, followed by stable convergence after approximately 1,000,000 steps. The trained agent achieves an average level completion time of 29.8 seconds, with a success rate of 88% and a generalization capability of 72% providing a replicable foundation for academic and professional applications.