Home / Summaries / Knowledge Management and Business Intelligence / markov-probabilities-downgrades

Data Science Applications - Marketing Analytics

4 important questions on Data Science Applications - Marketing Analytics

What are Markov Chains?

We start from a collection of random variables which can take values in one of M states. Once the current state is known, history may be thrown away. It is a memoryless random process with probabilities to change state.

The transition probabilities can be obtained using the idea of maximum likelihood with p(i,j)=n(i,j)/n(i)

Markov simplifies simulations and often comes close to reality.

Sum across rows is always 1, not across columns

What is the Markov Reward Process?

A cost for upgrading or discount for downgrading. Each membership gives a different reward.

V = sum[(T/(1+d))^t * R)

This can also be used to go to infite time horizon. Then V becomes = (I - 1/(1+d) * T)^-1 * R

What is the Markov Decision Process Approach?

Problem: find optimal policy pi for decision makers where pi(s) specifies best action in state s

Solved by:
- brute force evaluation
- value iteration
- policy iteration
- dynamic programming

Mover stayer model: stayers never leave, movers make transitions according to Markov chain with transition matrix
Can also be modelled via a cumulative model

What are the pros and cons of the Markov Chains?

Can be used for both existing customers and prospects. Easy to understand, flexible and based on solid theory.

However, the markovian property sometimes questioned
- downgrades tend to be more easily followed by further downgrades
- duration dependence effect: the lower a customer keeps same state, the lower migration probability
- migration probabilities tend to be correlated with business cycles

Therefore also Hidden Markov models are made. That uncover states of customer behavior