Proof of Principle Experiments: A Rediscovery Task

A "Rediscovery Task" was used to determine the performance of the Robot Scientist. This involved rediscovering the functions of ORFs from the well studied Aromatic Amino Acid Biosynthesis (AAA) pathway of S. cerevisiae, shown in figure 1. Of the 16 ORFs involved in the AAA pathway, only 8 were found to be auxotrophs, these were YBR166C, YDR007W, YDR035W, YDR354W, YER090W, YGL026C, YKL211C and YNL316C. Of the 22 metabolites involved in the pathway, only 9 were available for experimentation, these are shown in table 1 along with their relative costs.

Figure 1: The Aromatic Amino Acid Biosynthesis pathway of S. cerevisiae

Table 1: Relative Costs of Metabolites available for the Rediscovery Task

Knockout mutants corresponding to the 8 auxotrophic ORFs were created and made available to the robot, as were the 9 metabolites, with a concentration of 0.2 mg/l. Considering multiples of the 9 metabolites, a total of 2^9 i.e. 512 experiments are possible for each of the 8 ORFs, giving a total experiment space of 4096. However the study was limited to single and pairs of metabolites i.e. 45 combinations per ORF, or 360 possible experiments in total. The capacity of the robot was limited to considering 1 experiment for 4 ORFs per 48 hour period (c 6hrs preparation by the robot and 24+ hrs growth time for the knockouts. Exhaustive experimentation would therefore require more than 180 days, given that experiments could be interleaved: one set of experiments for 4 ORFs being prepared while experiments for the other 4 ORFs are being incubated.

However, simulations of the experimentation showed that solutions for most of the knockouts could be acquired in 5 iterations (i.e 10 days), so the maximum allowable number of experiments was set at 5 (MAXI).

As well as a proof of principle study for the Robot Scientist, these experiments were also designed to evaluate the performance of the intelligent experiment selection strategy, by comparing it with choosing experiments at random, and with a nave strategy that always chooses the cheapest available experiment. Simple algorithms for these strategies are given below:

Select and execute 
t_min from T


I=1 


REPEAT


  IF (H=S or I=MAXI) 
  THEN FINISH
  ELSE IF H is empty 
    THEN
      Select and execute 
         t_min from T
    ELSE
      Select and execute 
         t_minEC from T 


  I = I + 1


UNTIL FINISH

I=0 
 
REPEAT 


  IF (H=S or I = MAXI) 
  THEN FINISH 
  ELSE 
    Select and Execute 
       t_random from T 


  I=I+1 


UNTIL FINISH

I=0 


REPEAT 


  IF (H=S or I=MAXI) 
  THEN FINISH 
  ELSE 
    Select and Execute 
      t_min from T 


  I=I+1 


UNTIL FINISH

where H is the set of currently valid hypotheses, S is the set of hypotheses corresponding to the solution, T is the set of currently available trials, t_min is the cheapest available trial, t_minEC is the trial that minimises the expected experimental cost, t_random is a trial randomly chosen from T, I is the current iteration and MAXI is the maximum allowable number of iterations. After a particular trial is executed by any of the strategies, that trial is removed from T.

Figure 2: Classification Accuracy vs Iterations for Random, Nave and Ase Experiment selection methods (Robot)

Figure 3: Classification Accuracy vs Log Experimental cost for Random, Nave and Ase Experimental selection methods (Robot)

Figure 4: Classification Accuracy vs Iterations for Random, Nave and Ase experimental selection methods for 0% and 25% noise (Simulation)

Figure 5: Classification Accuracy vs Log Cost of Experimentation for Random, Nave and Ase experimental selection methods for 0% and 25% noise (Simulation)

Table 2: Enzymes, ORFs and Genes that participate in the AAA pathway of S. cerevisiae

Table 3: Solutions for the AAA pathway Rediscovery Task, mappings from ORF to enzyme(s). Multiple solutions indicate that the necessary metabolites for any further refutation of hypotheses were not available