MakeFigure3

   Generates Figure 3.

   Simulates a POMDP on one of the results obtained in the experiment in
   Figure 1. If the results are not pre-computed we compute them on the
   fly (this can be quite time consuming).
   

   The simulation shows the evolution of the belief and the
   executed actions and the obtained reward are printed in the Matlab
   console. 
   At the end we also print the accumulated discounted reward for the
   trial.
   This data can be used to generate a sequence of snapshots as
   those in the figure.

   This is an example of how to use the resulting policy out of a planning
   experiment.

   Parameters
     n: Number of steps of the simulation. If not given 15.

 'n' == simulation steps

0001 function MakeFigure3(varargin)
0002 %   Generates Figure 3.
0003 %
0004 %   Simulates a POMDP on one of the results obtained in the experiment in
0005 %   Figure 1. If the results are not pre-computed we compute them on the
0006 %   fly (this can be quite time consuming).
0007 %
0008 %
0009 %   The simulation shows the evolution of the belief and the
0010 %   executed actions and the obtained reward are printed in the Matlab
0011 %   console.
0012 %   At the end we also print the accumulated discounted reward for the
0013 %   trial.
0014 %   This data can be used to generate a sequence of snapshots as
0015 %   those in the figure.
0016 %
0017 %   This is an example of how to use the resulting policy out of a planning
0018 %   experiment.
0019 %
0020 %   Parameters
0021 %     n: Number of steps of the simulation. If not given 15.
0022 %
0023   % 'n' == simulation steps
0024   
0025   if nargin==0
0026     n=15;
0027   else
0028     n=varargin{1};
0029   end
0030 
0031   fprintf('Loading/Generating the simulation results\n');
0032 
0033   GenData=@()(TestRep('Test1','Figure2',1));
0034   Results=GetData('Results/Test1-Figure2-1.mat',GenData);
0035   
0036   % Get the parameters used to generate the results
0037   P=Results.P;
0038   
0039   % Get the POMDP
0040   POMDP=Results.POMDP;
0041   
0042   % Get the las policy of the simulation
0043   Policy=Results.V{end};
0044   
0045   % Get the initial belief from the parmeters
0046   b=P.start;
0047   
0048   % The hidden state is selecte at random
0049   s=rand(b);
0050   
0051   fprintf('SIMULATING\n');
0052   fprintf('INITIAL STATE %f\n',s);
0053    
0054   h=clf;
0055   set(h,'name','C-POMDP Figure 3','numbertitle','off');
0056   
0057   h=plot(b); 
0058   xlabel('s');
0059   ylabel('p(s)')
0060   axis([-20 20 0 1]);
0061   hold on;
0062   rAll=0;
0063   gamma=get(POMDP,'gamma');
0064   g=1;
0065   for i=1:n
0066     fprintf('\nSTEP %u: ',i);
0067     a=OptimalAction(Policy,b);
0068     fprintf('  action: %u (',a);
0069     switch a
0070       case 1
0071         fprintf('Move-Left');
0072       case 2
0073         fprintf('Move-Right');
0074       case 3
0075         fprintf('Enter-Door');
0076     end
0077     [s b o r]=SimulationStep(POMDP,b,s,a);
0078     rAll=rAll+g*r;
0079     g=g*gamma;
0080     fprintf(')  state:%f  reward: %f   ',s,r);
0081     delete(h);
0082     h=plot(b);
0083     fprintf('<press a key>');
0084     pause;
0085   end
0086   hold off
0087   fprintf('\n\nDiscounted reward: %f\n',rAll);
0088   
0089   
0090