Research & Project Work

Methodology

The project proceeds in two mathematical stages. The first formulates navigation as a discrete Markov Decision Process on a grid, with the agent's observations defined egocentrically — relative to its own heading — rather than in absolute coordinates; this is the mechanism through which viewpoint-dependence is introduced. The second reformulates the same problem as a continuous dynamical system, discretised numerically (Euler / Runge–Kutta), in which the trained policy induces a closed-loop vector field analysable with standard dynamical-systems techniques. Environments are implemented first in Python, then extended into an interactive 3D visualisation built with the Ursina library, drawing on the parametric-animation and 3D-kinematics techniques developed in prior undergraduate work (A Python-based Approach to Animate Geometry and Data). Reinforcement-learning agents (Gymnasium environments, PyTorch models) are trained under a canonical landmark configuration and evaluated under controlled perturbations — landmark displacement, removal, and geometric transformation — with performance compared across agents of differing memory capacity using multi-seed statistical testing.

Objectives

• Formalise spatial navigation as a discrete Markov Decision Process, then extend it to a continuous dynamical system governed by z = (x, y, θ), u = (v, ω), ż = f(z, u).
• Design a controlled simulation environment — a 2D grid initially, later a continuous 3D environment — in which landmark geometry and agent viewpoint can be manipulated systematically and independently.
• Train reinforcement-learning agents of varying memory capacity (memory-less, short-history, recurrent) using Gymnasium and PyTorch.
• Test whether egocentric perception of an asymmetrically placed landmark, rather than the presence of an opening alone, is what produces entrance/exit navigation asymmetry.
• Analyse trained agents' closed-loop trajectories using dynamical-systems tools: equilibrium analysis, linearisation, and empirical estimation of sensitivity to initial conditions.
• Build a 3D visualisation pipeline, extending parametric-animation methods developed in prior undergraduate work, to render agent training and spatial dynamics interpretably.