**Understanding Markov Chains Examples And Applications Pdf** – The Markov assumption states that the past does not provide valuable information. Looking at the present, history has nothing to do with the future.

The Markov chain gives a new dimension to probability theory. Application can be found in almost all fields. Also, many ideas of value efficiency have been developed based on the Markov chain; Their importance is very important in the field of data analysis.

## Understanding Markov Chains Examples And Applications Pdf

Being an engineering student, I was wondering how I could pass some subjects after spending a week with them before the final exam. Despite spending 3-4 hours a day, I found nothing during the session. Until one week before the semester exams, my way of learning those subjects was bumpy. I thought I would fail. But from the final result, I thought how I made a difference in just one week of the exam.

## Metropolis Hastings And Bayesian Inference

A few years ago, when I started my journey in data analysis, I came across a theory called the Markov chain. This answer and explanation provided what I was looking for in a default scenario. There was an important property of Markov chains that caught my attention at the time.

After this, I began to delve deeper into the spiral of the Markov chain. I will try to simplify the concepts with the help of real life examples. I know the content is long and takes time to read. But you know what I found while going through the various materials. It’s full of high-level math. From my teaching experience, I think you need to have an intuition of the subject first to understand what examples you can come up with, then do the math and then code. This content is a mix of more examples and a bit of math. I hope you enjoy it.

A century ago, Russian mathematician Andrei Andreyevich Markov invented a completely new branch of probability theory. Well, it has an exciting story. While studying Alexander Pushkin’s novel Eugene Onegin, Markov spent hours sorting out patterns of vowels and consonants. On January 23, 1913, he summarized his discovery in an address to the Imperial Academy of Sciences in St. Petersburg. His findings did not change the perception or evaluation of Pushkin’s poetry. However, the technique he developed, now known as Markov chains, extended probability theory in a new direction. The Markov method goes beyond coin-flipping and dice-rolling scenarios (each event is independent of the others) to chains of linked events (what happens next depends on the current state of the system).

In a Markov chain, the future depends only on the present, not the past. Let’s dive into it. According to Wikipedia, “A Markov chain or Markov process is a stochastic model that describes a sequence of possible events, where the probability of each event depends only on the state obtained during the previous event.”

#### Markov Chain Process (theory And Cases)

To me, most of the time we confuse the words casual and random. We often say “random means random”. To explain this better, let me give you an example. Let’s say I’m in a pub and I’m having fun with a friend drinking beer and playing a game. We set a rule. I’m drinking a few bottles at the moment; My boyfriend drinks twice as much as I do at the time. How to drink a bottle in a minute. Drinks two bottles in one minute. Or if I drink two bottles in one minute, he drinks four bottles in one minute. I can define it as follows.

So no coincidence. It is a combination of processes that contains at least one random process. And the decision making process may or may not be based on the stochastic method. I hope I made it clear. If you are confused about what a deterministic approach is, let me give you an example. The sun always rises in the east. This is a decisive statement, a universal truth. The sun never rises in the west. Ah, but don’t look at the contest I mentioned earlier. Alcohol is harmful to health. This is the hypothetical situation I mentioned.

The philosophy behind Markov chains is the understanding of correlation, which is an interesting aspect of real-life problem solving. Those familiar with the “domino effect” know that when you change a behavioral trait, a chain reaction is set in motion and the corresponding behavioral traits change as well. Our real world is full of such examples. Please check the examples below.

A series of chain reactions developed as bad as the First World War. Unfortunately, Markov is still working on his research. Who knows, maybe he would bring it to the world and if the world’s mathematicians and statisticians used Markov chains at one time, we would be less affected. In all industries like stock market, banking, IT, etc., the “domino effect” is always happening. Our job as data scientists is to minimize its impact. So we do. The advent of computers and knowledge of ML and DL helped us avoid the danger of stochastic systems. And the first tool to start the journey is the Markov Chain.

### Markov Decision Processes With Applications To Finance

Now let’s explore some mathematical concepts. I’ll try to make it less boring for you. It’s even better if you have several

Basic intuition for understanding probability theory, linear algebra, and graph theory. Please don’t panic. When we talk about a “chain”, we get the idea of a “graph”. The next question arises: is it cyclical or one-sided? Let me rephrase my previous example of drinking beer in a tough competitive way. At this point I told my friend

I’ll drink three times as many bottles of beer as you will” is a challenge, right? Below diagram for better intuition. The graphs below basically show the Markov chain we might be dealing with

Now I’ll give you an idea of the recent past with an example of my competitive pub games controlling the current outcome. I will not explain it. Looking at the picture is self-explanatory.

### Classification Of States

I hope the example above gives you some insight into the process. One thing that might surprise you is that Markov chains don’t talk about any “past”. But why did I mention the words “recent past” and “distant past”? But I think you can develop strong conceptual intuition just by thinking about it. Think “immediate past” is “yesterday” and “distant past” starts from the day before “yesterday”, trust me, it will help you more. Like you learned a lot yesterday, got good results today or got inspired “today” because you met someone “yesterday”. Let me give another example of a real life problem. Before that, let’s define a Markov chain in terms of probability. The Markov chain determines three factors.

· State season(s). The country’s seasonal states are defined as summer, monsoon, autumn, winter and spring. So in seasonal state-space we refer to the five seasonal states.

If the transition operator does not change during the transition. A Markov chain is called “time homogeneous”. So, T

à∞ The chain approaches an equilibrium known as a stationary distribution. For this, the following equation appears.

## Find The Probability Of A State At A Given Time In A Markov Chain

Now let’s come to another example. You may have noticed that the Gmail editor always suggests the next word when you type a message. I have seen this to be true in my case many times. It is an application of Markov Chain in the field of Natural Language Processing (NLP). More precisely, it uses the concept of “n-gram”. Our example is based on it. Consider the following three sentences.

Here we consider the concept of uni-gram. We hope the following image gives you an idea of the n-gram concept.

. A “path”, called the trajectory of a Markov chain, is synonymous with a “path” in a graph. For better understanding, condition and process are same. So, X

I also define the transition diagram of the Markov chain. It is a single weighted directed graph, where each vertex represents a Markov chain state and a directed edge leading from vertex j to vertex i (from one state to another or the same state) if the transition probability is p.

## Using Higher Order Markov Models To Reveal Flow Based Communities In Networks

So you may be wondering how many states are possible. An infinite number of states are theoretically possible. This type of Markov chain is called a continuous Markov chain. But when we have a limited number of states, we call it