This textbook will present, in a rigorous way, the basic theory of the discrete-time and the continuous-time Markov chains, along with many examples and solved problems. This textbook is intended for a basic course on stochastic processes at an advanced undergraduate level and the background needed will be a first course in probability theory.