Against observed phenotypes
Source:vignettes/articles/phenotype-validation.Rmd
phenotype-validation.RmdWhen a strain is observed to use a substrate, does its genome support the corresponding GIFT? We compared curated genomic calls with strain-resolved BacDive assays, growth on chemically defined MediaDive media, and a second, species-level trait compilation. The result is a test of where the genomic evidence reaches an observed capability, not a prediction of what a strain will do in any particular assay.
These results are pinned to database 2026.27.1 and its committed analysis tables. The live catalogue can contain additional GIFTs or changed routes.
In short
- Recall is reported separately for each observed phenotype. An EC-resolved enzyme assay tests a reaction; substrate use and product formation test a GIFT.
- A positive genomic call with no observed phenotype is allowed: expression, regulation and assay conditions lie outside the GIFT claim. An observed capability with no supported call is the useful disagreement to investigate.
- Only 17 of 153 GIFTs have a direct individual recall test at this threshold. Another 44 belong to bounded anabolic frames: defined-medium growth can challenge their nutrient-level support, with limits explained below. The remaining 92 have no usable test in these references.
What was compared
BacDive records measured strain phenotypes and assembly accessions.
We joined those accessions to the reference genomes, mapped metabolite
identifiers to declared anchors and EC identifiers to reactions, and
excluded BacDive’s genome-based predictions. The
reviewed crosswalk admits a phenotype to a recall calculation only where
its boundary is equivalent to, or narrower than, the GIFT
claim. For example, gelatin hydrolysis is not treated as an observation
of collagen cleavage, and motility is not treated as an observation of a
flagellar apparatus.
For an observation that passes that rule, recall is the share of
observed-positive records whose matched genomes support
the target. The n beside each point below counts
all paired observations, positive and negative; the recall
denominator is the observed-positive subset. Each row also gives the
number of represented genera. Rows are kept separate even when they name
the same substrate, because growth on a sugar and acid production from
it are different assays.

BacDive recall by curated observation, with all paired records (n) and represented genera shown at right. Only equivalent or subset-of mappings with at least 20 pairs appear. The positive subset of each n is the recall denominator. No rate is pooled across observations.
Urease activity reaches 0.866 recall across 1,012 paired records; urea utilisation reaches 0.822 across 661. Indole production reaches 0.721. N-acetylglucosamine use reaches only 0.205: tracing its missing reaction led to curation of a second route through the phosphotransferase system. That is a specific route lead, not evidence that the phenotype itself was inferred from the genome.
A disagreement has a direction
The 1,569 supported-call/negative-assay comparisons are permitted under the GIFT claim. Of the 705 observed-positive/unsupported-call comparisons, 489 have a closest route with a named missing requirement and 216 have no genomic evidence for that capability. These are counts of genome–observation comparisons, not distinct genomes. A missing requirement is a curation lead; it is not automatically an annotation error or a missing enzyme.
Two assays in BacDive sometimes disagree with each other. For urease activity versus urea utilisation, 560 of 563 genome-backed strains agree. For tryptophanase activity versus indole production, only 321 of 351 agree. An unexpectedly low recall on one assay needs to be read beside that assay’s own consistency with the reference.
A second reference checks the mapping
Madin’s published species-level compilation records carbon-substrate use. It offers an independent check on whether BacDive’s growth and acidification rows should be kept separate. The comparison below uses the reported recall for each source; the strain and species denominators differ, so the columns are not paired measurements of the same organisms.
| Substrate | BacDive growth/use | BacDive acidification | Madin substrate use |
|---|---|---|---|
| L-arabinose | 0.294 | 0.562 | 0.309 |
| D-galactose | 0.557 | 0.759 | 0.559 |
| L-rhamnose | 0.098 | 0.667 | 0.180 |
| D-xylose | 0.769 | 0.816 | 0.662 |
The independent substrate-use values track BacDive growth/use more closely than acidification. Madin requires a species-to-genome representative choice; the analysis repeated that choice 100 times and recorded its sensitivity. Its motility row, like BacDive’s, is excluded from flagellar-apparatus recall because motility does not imply a flagellum.
What defined media and annotation add
MediaDive connects growth-positive strains to chemically defined media. If a medium omits a biomass-essential nutrient and the strain grows, that is an observed-positive test of the relevant biosynthetic capability. Alternative curated routes to the same nutrient are OR-ed: growth without cysteine asks whether the genome supports either curated cysteine route, not whether it supports both. Only media whose ingredients are at least 75% resolvable to identifiers enter this inference. For each growth-positive strain–medium pair, the analysis records which omitted nutrients have genomic support; it then summarises support by nutrient and by genome within a bounded frame. An unsupported product points to GIFT evidence worth tracing. A medium that supplies a nutrient gives no test of its synthesis.
This gives the strongest indirect check for 26 of the 44 frame-only GIFTs: they make 24 amino-acid or nucleotide products. Across those products, the committed table records 5,803 nutrient-level tests and 0.844 recall. The per-nutrient table shows where support is strong or weak; the disagreement table retains the genomes, media and omitted nutrients behind unsupported calls.
The other 17 measured frame-only GIFTs make cofactors or inorganic
nitrogen products. Their nutrient-level rows are reported, but a strain
may grow without making a particular cofactor or without fixing
nitrogen, so a low support fraction is not by itself a failed genomic
call. The final frame member, siroheme_to_heme_b, was
excluded from this test because the KO-only genome annotation cannot
evidence one of its components. Thus 44 counts frame
membership, not 44 independent phenotype assays or 44 equally
falsifiable GIFTs. Even where a nutrient test is sound, growth implies
some way to obtain that product; it does not identify which curated
route the organism uses.
The primary comparison uses KEGG’s own KO assignments. On the same 398 genomes, a pinned annotation pipeline recovered 0.906 of the complete KEGG-route calls using KofamScan alone, and 0.963 after adding NCBIfam, dbCAN and Pfam evidence. Those fractions describe the change in total supported calls, not per-GIFT recall or overall accuracy; some capabilities gained calls while others lost them. Gene calling was held fixed, so this comparison isolates marker assignment rather than all losses a MAG annotation workflow could introduce.
How much of the catalogue could be tested?
For this 153-GIFT release, 17 GIFTs had a direct individual recall test at 20 or more paired observations. Another 44 had no such assay comparison but belonged to bounded anabolic frames; 43 entered the indirect defined-medium analysis described above. The other 92 had no usable recall test in these references: some had no public observation, while others had too few records or only a related or refused proxy. This is a limit of the available observations and crosswalk, not a negative GIFT call.
The coverage table names the status of every GIFT. The full manuscript account sets out the comparison and its limitations.
Reproduce this analysis
All figures and values on this page read the committed R9
tables. The producing scripts are 03-phenotype.R,
04-auxotrophy.R,
05-annotation-route.R,
07-madin.R
and the coverage
audit. The reviewed phenotype
crosswalk states exactly which observation licenses each
comparison.