Documentation

About the package

This package is primarily based on the quasistatic version of the “model switch integration method” laid out in a paper by Nicolas Lartillot and Hervé Phillipe titled Computing Bayes Factors Using Thermodynamic Integration

The modification we have made to their method is changing the value of beta at every step of the MCMC chain rather than at set intervals. Beta is therefore changed very slowly, so as to not cause issues with equilibration.

Summary of the method

The main idea of this method is to progressively change the target density from being the posterior of one model to the posterior of another model, which allows estimation of the Bayes factor.

This is accomplished by setting the posterior density to the product of the two model posteriors, with one raised to the power of the parameter beta and the other raised to 1 - beta.

Regularly across the chain, a value called U in the Nicolas Lartillot and Hervé Phillipe paper is calculated, which is later used in estimating the Bayes factor.

Please read the above paper to obtain a more thorough understanding of this method.

Installation

Please refer to the instructions for installation

Basics of Usage

This package can be used to calculate the Bayes factor of one model over another. It can be used in certain cases where path sampling fails, particularly where sampling from the prior is impractical.

One use of this is for structured trees (eg. using MultiTypeTree) which becomes challenging when there are more than just a few demes.

It is also worth checking out the tutorials.

General overview of how this package can be used:

  1. Install the ModelComparison package
  2. Setup a BEAST XML file which specifies your analysis (see tutorials and below documentation, as well as examples in the package directory)
  3. Equilibrate the run (run an initial chain until equilibrated) (more detail in tutorials)
  4. Modify the betaControlMode value in the XML file (telling ModelComparison to progressively switch between your two models)
  5. Resume the run using the modified XML file
  6. Run the command line tool (built-in) to estimate the Bayes factor using the log file

Overview of the package

This package uses:

BEAST Objects

beast.math.distributions.ModelComparisonDistribution

This BEAST object is a Distribution, and it does the work of calculating the posterior using the beta value and the two provided Distribution inputs.

Options/Inputs

name:betaParameter

This is a RealParameter (ie. a decimal number) which is used by ModelComparisonDistribution in calculating the posterior.

(This should be set to either 0 or 1 in your XML)

name:distribution

You must provide exactly two distributions as input to the ModelComparisonDistribution. These will be your posteriors from your two models which you are wanting to compare.

The first distribution will be raised to the power of (1 - beta) in the posterior, and the second distribution will be raised to the power of beta.

Usage example

<distribution
id="posterior"
spec="beast.math.distributions.ModelComparisonDistribution" 
betaParameter="@myBetaParameter">
	<distribution id="posterior_for_first_model"/>
	<distribution id="posterior_for_second_model"/>
</distribution>

beast.core.ModelComparisonMCMC

This extends BEAST'S usual MCMC chain code to allow for the values of beta to be changed very slowly at every step of the chain.

Options/Inputs

name:betaParameter

This must be the same betaParameter which is used by ModelComparisonDistribution as above.

name:betaControlMode

This is a text input which determines how the ModelComparison package should control the value of the beta parameter (if at all) so that it slowly changes over the course of the BEAST run.

Valid options:
statickeep beta the same throughout the run
onewayslowly change beta from either 0 to 1 or 1 to 0 throughout the run
bothwaysslowly change beta in one direction, then back again to the starting value throughout the run

Note: Before using Model Comparison to control the value of beta, you must first equilibrate your run using beta set to either 0 or 1

Usage example

Instead of using this: 
<run chainLength="20000000" id="mcmc"
spec="beast.core.MCMC">
</run>
Use this: 
<run chainLength="20000000" id="mcmc"
spec="beast.core.ModelComparisonMCMC"
betaParameter="@myBetaParameter"
betaControlMode="static">
</run>

beast.core.util.ModelComparisonLogger

This allows for logging the values of beta and U (as defined in the Lartillot and Phillipe paper).

Options/Inputs

name:posteriorDistribution

This is the ModelComparisonDistribution which is your posterior distribution.

Usage example

Add the following to any logger (eg. the screenlog or tracelog logger: 
<log spec="util.ModelComparisonLogger" posteriorDistribution="@posterior"/>
<!-- @posterior here means that our posterior object has the id "posterior" -->
Or alternatively add a new logger which saves only the beta and U values to a file: 
<logger fileName="$(filebase)_beta_U.log" id="betaAndUValueLogger" logEvery="500">
	<log spec="util.ModelComparisonLogger" posteriorDistribution="@posterior"/>
	<!-- Here we have used $(filebase) in the filename - BEAST will replace that part with the filename of your XML file (without the .xml extension) -->
</logger>

beast.app.tools.ModelComparisonCalculator

This is a command-line tool for analying the log files outputted with a ModelComparisonLogger included.

It takes your log file and produces an estimate for the Bayes factor.

See this guide to using the calculator