You can apply continuous-time reinforcement learning to complex, memory-dependent systems by first converting them to Markovian approximations—this opens up RL applications to event-driven processes like financial markets and network dynamics.
This paper tackles reinforcement learning for systems driven by Hawkes processes—mathematical models where events cluster in time and influence future events. The key challenge is that these systems have memory and aren't Markovian, so standard RL doesn't apply.