### Abstract

Phylogenetic tree reconstruction is a fundamental biological problem. Quartet trees, trees over four species, are the minimal informational unit for phylogenetic classification. While every phylogenetic tree over n species defines (_{4}^{n}) quartets, not every set of quartets is compatible with some phylogenetic tree. Here we focus on the compatibility of quartet sets. We provide several results addressing the question of what can be inferred about the compatibility of a set from its subsets. Most of our results use probabilistic arguments to prove the sought characteristics. In particular we show that there are quartet sets Q of size m = cn log n in which every subset of cardinality c′n/ log n is compatible, and yet no fraction of more than 1/3 + ε of Q is compatible. On the other hand, in contrast to the classical result stating when Q is the densest, i.e., m = (_{4}^{n}) and the compatibility of any set of three quartets implies full compatibility, we show that even for m = Θ ( (_{4}^{n}) ) there are (very) incompatible sets for which every subset of large constant cardinality is compatible. Our final result relates to the conjecture of Bandelt and Dress regarding the maximum quartet distance between trees. We provide asymptotic upper and lower bounds for this value.

Original language | English (US) |
---|---|

Pages (from-to) | 1493-1507 |

Number of pages | 15 |

Journal | SIAM Journal on Discrete Mathematics |

Volume | 28 |

Issue number | 3 |

DOIs | |

State | Published - Jan 1 2014 |

Externally published | Yes |

### All Science Journal Classification (ASJC) codes

- Mathematics(all)

### Keywords

- Phylogenetic reconstruction
- Quartet amalgamation
- Quartet fit
- Tree compatibility

## Fingerprint Dive into the research topics of 'On the compatibility of quartet trees'. Together they form a unique fingerprint.

## Cite this

*SIAM Journal on Discrete Mathematics*,

*28*(3), 1493-1507. https://doi.org/10.1137/130941043