r,for-loop,nested-loops,markov-chains,markov

Your y <- y+1 is in one too many curly brackets. It is not being called most of the time. I think this is what you want: x1 <- runif(100, 1.0, 100) markov <- matrix(c(.5,.3,.2,.4,.5,.1,.2,.4,.4),nrow=3) m1 <- markov[1:3] m2 <- markov[4:6] m3 <- markov[7:9] y <- 2 x <- 1...

artificial-intelligence,markov

You're on the right track with those equations. You just need to consider each of the four possible policies in turn: (slow, slow), (fast, slow), (slow, fast), (fast, fast). Consider (slow, fast): From a) you have already seen J*(cool) = 40. J*(warm) = 10 + 0.9 * (0.875 * J*(warm)...