Evaluate a community of genomes from one annotation table

Evaluates every genome of a multi-genome annotation table separately and binds the calls into a [gifter_community()]. Each genome is evaluated by [evaluate_gifts()] on its own markers, against one database release; genomes are evaluated in parallel because they are independent of one another.

Usage

evaluate_gifts_community(
  annotation_table,
  namespace = NULL,
  db = NULL,
  genome_id = NULL,
  gene_id = NULL,
  max_genes = 5000,
  workers = NULL,
  progress = NULL
)

Arguments

annotation_table: A data frame of markers for several genomes, with `genome_id`, `namespace` and `accession` columns and, normally, `gene_id`.
namespace: Namespace for a character-vector input. If omitted, KO, EC, Pfam, TIGRFAM, NCBIfam and CAZy accessions are inferred where possible.
db: Optional open gifter database connection.
genome_id: Name of the column separating genomes. Defaults to `genome_id`. Every row must name a genome.
gene_id: Which column names genes, resolved as in [evaluate_gifts()] over the table as a whole. The genome column is never proposed for it.
max_genes: Number of distinct gene identifiers above which a single genome is questioned, applied to each genome separately. `Inf` evaluates every genome without asking.
workers: Number of genomes to evaluate at once. Defaults to the `mc.cores` option, or to one fewer than the number of physical cores, and is capped at the number of genomes.
progress: Whether to display a progress bar. Defaults to `TRUE` at an interactive console evaluating more than one genome, and to `FALSE` otherwise. The display never changes a call.

Value

A `gifter_community` list. Its `results` member holds the `gifter_genome` result of every genome, under its genome identifier.

Why a community is not one evaluation

Markers pooled across genomes complete routes that no member encodes, so the pooled call is a statement about the collection and not about any genome in it. That is what the single-genome guardrail of [evaluate_gifts()] refuses, and this function is the way past it: the genome column is what separates the markers, and the evaluation never sees two genomes at once. A genome evaluated here is identical to the same genome evaluated alone.

The genome column is required and is never inferred from column order. A misread gene column mislabels evidence inside a genome, which is why one may be proposed for approval; a misread genome column redraws the genomes themselves, and no call in the result would look wrong afterwards.

Arguments are checked before anything is evaluated

Evaluating a large community takes minutes or hours, so everything that can be decided from the annotation table is decided before the first genome is evaluated: the genome column, the gene column, the shape of the `workers` request, and the single-genome guardrail of every genome. A malformed argument therefore costs a message rather than the run.

Nothing about how the calls will later be read is settled here. Genome abundances and completeness estimates belong to [community_traits()], so a community can be evaluated once and read under several thresholds.

Parallel evaluation

Before the first genome is evaluated, a SQLite database is copied with SQLite's online backup API to a run-specific, read-only snapshot. Sequential evaluation and every forked worker read that same snapshot. Rebuilding or replacing the original database during a run therefore cannot mix releases, and the temporary snapshot is removed on success, error, or interruption. There is no retry against a newer database. A non-SQLite custom DBI connection cannot be copied and is evaluated sequentially on that one open connection.

Parallelism changes wall time only: the calls, their order, and the assembled community are the same at any number of workers. Forking is unavailable on Windows and for a connection with no file behind it, where the genomes are evaluated one after another.

A community of any size reports its progress at an interactive console, in genomes evaluated out of genomes to evaluate, with an estimate of the time remaining. The count advances genome by genome whether one worker or many produced them, so it measures the run rather than the way the work was spread out. Nothing is reported when there is nobody watching, which is why a script, a knitted document and a package check stay silent unless they ask for the display with `progress = TRUE`.

Examples

markers <- data.frame(
  genome_id = c("A", "A", "B"),
  gene_id = c("g1", "g2", "g1"),
  namespace = "KO",
  accession = c("K01198", "K01805", "K01805")
)
community <- evaluate_gifts_community(markers, workers = 1)
community$genome_id
#> [1] "A" "B"