Cyclic Nature of Tree Generation Process

These plots were generated for the small graph problem of 6 nodes and 8 arcs. I wanted to investigate whether iteratively generating the decision tree would eventually lead to convergence. As shown on the left this procedure does not converge, but rather cycles periodically. These plots are for a specific epsilon, but I have verified that similar results are seen for other values of epsilon. Rather than sampling a subset of the 256 world states, I used the entire set to generate the next decision tree. Each of these plots used different starting decision trees as the initializer. The initial decision tree is generated randomly by sampling with replacement assuming no computation costs.

In these plots it is seen the that the minimum cost occurs during the transient section. For other values of epsilon this varies.