Markov defined a way to represent real-world stochastic systems and processes that encode dependencies and reach a steady-state over time.