Corrected Results for QL on New Graph

These are the first results that look like they make sense for the QL MDP model for the new graph. The one suspicious feature is that the MDP model does not show as dramatic an improvement in this case. Either there is an error or one possibility may be that the number of samples made to determine the transistion probabilities were not enough. I generated 1000 random graphs in order to gather transition probabilities, but there are at least 4096 possible graphs. So I am undersampling. I have various data sets which I will try. These results are for data set 3, where value iteration took 27 sec per iteration, approximately 15 iterations on average. So for 20 different values of computational cost this run took 2.25 hours. Each state is updated during each iteration. I put some logic in there to skip some states with not outgoing transitions to speed up the code. I have another data set that I will try next.
As a sanity check:
It seems to do the right thing when computational costs are high. When computation cost is zero there is s problem. I will investigate.
I also learned that I was handling the sparse matrices incorrectly. In my code I was calculating the transition probability by summing up rows and dividing each column of that row by the sum. The problem was that I was using the find function in Matlab and was repeating the same calculations for each row. The calculations are correct, but I was doing twice the work.
Even so, I think that using sparse matrices will slow down the code, but not to the extent that was happening before