Genomic intervals¶

Composition-matched backgrounds, motif proximity to genomic anchors, hits in genomic coordinates, and ranges passed to and from PyRanges and polars-bio. Positions count from 0 and leave the end out, as in BED files, PyRanges and Python slices, and ranges can be given as DataFrames or tuples. PyRanges objects, pandas DataFrames, including those with PyRanges' Chromosome, Start and End columns, and polars-bio frames can be given wherever ranges are taken; polars-bio frames marked as counting from 1, as polars-bio makes them by default, are converted.

match_bkg ¶

match_bkg(
    sequences: Sequence[str] | Mapping[str, str],
    universe: Sequence[str]
    | Mapping[str, str]
    | None = None,
    genome: Mapping[str, str] | None = None,
    exclude=None,
    n_candidates: int | None = None,
    chromosomes: Sequence[str] | None = None,
    n_per_target: int = 1,
    n_bins_gc: int = 20,
    n_bins_length: int = 10,
    covariates=None,
    universe_covariates=None,
    bigwig=None,
    target_ranges=None,
    n_bins_covariates: int | Mapping[str, int] = 10,
    bin_type_covariates: str
    | Mapping[str, str] = "quantile",
    unique: bool = True,
    seed: int | None = None,
    return_indices: bool = False,
)

Draw background sequences matched to targets on GC content and length.

HOMER-style matching: the universe is binned by GC fraction into n_bins_gc equal-width bins, by length into n_bins_length quantile bins and by any covariates, and each target draws n_per_target sequences from its own bin. When the bin runs short, neighbouring bins are searched ring by ring, up to two rings or one per axis; after that the remaining draws repeat sequences from the bins searched, with a warning. With unique, each universe sequence is otherwise drawn at most once.

Parameters:

Name	Type	Description	Default
`universe`	`Sequence[str] \| Mapping[str, str] \| None`	Candidate background sequences, as a list or a mapping from names.	`None`
`genome`	`Mapping[str, str] \| None`	Instead of `universe`, a mapping from chromosome names to sequences, such as a `Genome` on disk or a genome read with `read_fasta()`, to draw `n_candidates` random windows from; from a genome on disk, only the windows are read. The windows take their widths from the targets, and each target is matched among windows of its own width; they lie on `chromosomes`, which defaults to all of them, and avoid `exclude`, a range table such as the target peaks. `n_candidates` defaults to `max(10000, 10 * len(sequences))`.	`None`
`covariates`		Extra values to match on, one row per target and per universe sequence: DataFrames or mappings with the same columns, or arrays. `n_bins_covariates` and `bin_type_covariates`, which is `"quantile"` or `"equalwidth"`, apply to all of them or are given per name.	`None`
`universe_covariates`		Extra values to match on, one row per target and per universe sequence: DataFrames or mappings with the same columns, or arrays. `n_bins_covariates` and `bin_type_covariates`, which is `"quantile"` or `"equalwidth"`, apply to all of them or are given per name.	`None`
`bigwig`		With `genome`, match on bigWig signal as well, such as chromatin accessibility: `bigwig` is a path or a mapping from covariate names to paths, and `target_ranges` holds the coordinates of the targets in the same order, as a range table or (chrom, start, end) tuples. The mean signal over each target and each window, computed with `signal_stats()`, becomes a covariate binned as above. Needs the `bigwig` extra.	`None`
`target_ranges`		With `genome`, match on bigWig signal as well, such as chromatin accessibility: `bigwig` is a path or a mapping from covariate names to paths, and `target_ranges` holds the coordinates of the targets in the same order, as a range table or (chrom, start, end) tuples. The mean signal over each target and each window, computed with `signal_stats()`, becomes a covariate binned as above. Needs the `bigwig` extra.	`None`
`return_indices`	`bool`	Return a DataFrame of the matches instead, with 0-based `target_i` and `universe_i`, the GC fraction and length of each, any covariates, and `universe_name` when the universe has names.	`False`

Returns:

Type	Description
`list[str] \| dict[str, str]`	The matched sequences, `n_per_target` per target in target order, as a list or as a dict when the universe has names. In a dict, repeats get `.1`, `.2` and so on, and genome windows are named `chrom:start-end`.

Examples:

>>> import universalmotif as um
>>> targets = um.example_sequences("targets")
>>> controls = um.example_sequences("controls")
>>> bkg = um.match_bkg(targets, controls, seed=1)
>>> len(bkg), list(bkg)[:2]
(100, ['control_047', 'control_090'])

With return_indices, the matches can be checked:

>>> idx = um.match_bkg(targets, controls, seed=1, return_indices=True)
>>> idx.select("target_i", "target_gc", "universe_i", "universe_gc").row(0)
(0, 0.339, 46, 0.344)

motif_proximity ¶

motif_proximity(
    hits: DataFrame | None = None,
    anchors=None,
    motifs: Motif | Iterable[Motif] | None = None,
    sequences: Mapping[str, str] | None = None,
    universe=None,
    method: str = "binomial",
    count: str = "hits",
    orientation: str = "undirected",
    scan_pvalue: float = 0.0001,
    rc: bool = True,
    dist_thresholds: Sequence[int] | None = None,
    n_perm: int = 1000,
    perm_per_chrom: bool = True,
    ignore_strand: bool = True,
    qvalue: float = 0.1,
    seed: int | None = None,
    nthreads: int = 1,
) -> pl.DataFrame

Test whether motif hits sit closer to genomic anchors than chance.

The genome-wide counterpart of motif_peaks(). Each hit counts at its middle position. For each distance d in dist_thresholds, the near zone is every scanned position within d of an anchor; the default distances are 10, 25, 50, 100 and so on, up to half the smallest universe region. Three nulls are available:

"binomial" Hits uniform over the universe, so the number in the near zone is binomial; the best distance is Bonferroni-corrected for the distances tried. Fast, but blind to composition bias. "permutation" Anchors are placed at random in the universe n_perm times, keeping each on its own chromosome with perm_per_chrom. This absorbs composition and mappability bias, and is reproducible with seed. "ks" A one-sided Kolmogorov-Smirnov test of each hit's distance to the nearest anchor, for undirected zones and count="hits" only.

Parameters:

Name	Type	Description	Default
`hits`	`DataFrame \| None`	Hits from `scan_sequences()`, with `motif`, `score`, `start`, `end` and `strand` columns and the chromosome names in `sequence`. Give this, or give `motifs` and `sequences` as a mapping from chromosome names to sequences to scan them with `scan_pvalue` and `rc`; only chromosomes with anchors are scanned, and a `Genome` on disk is read one of them at a time.	`None`
`anchors`		The anchor coordinates, as a DataFrame with `chrom`, `start`, `end` or `width`, and optionally `strand` columns, or as (chrom, start, end[, strand]) tuples. Use 1 bp anchors for point hypotheses: wide anchors widen every zone.	`None`
`universe`		The regions scanned, as a range table or as a mapping from chromosome names to lengths for whole chromosomes. Required with `hits`; by default the whole of `sequences`. Zones are clipped at chromosome ends when lengths are known. Only chromosomes carrying anchors are used.	`None`
`count`	`str`	`"hits"` counts hits in the zone, where overlapping hits of a motif count once with the best score kept, across strands unless `ignore_strand` is false. `"anchors"` counts anchors with a hit nearby; its binomial null is a Poisson approximation, so the permutation test is preferred.	`'hits'`
`orientation`	`str`	`"undirected"`, or `"upstream"`/`"downstream"` of stranded anchors.	`'undirected'`

Returns:

Type Description

DataFrame

A DataFrame of motifs with q-values up to qvalue, in order of P-value, with the columns motif, motif_i, method, count, orientation, nhits, best_dist, hits_near, hits_far, frac_near, expected_near, enrichment, log2_enrichment, pvalue, qvalue, ks_d and dist_to_nearest. motif_i is the motif's 0-based index among motifs, or in hits the motif_i from scan_sequences() if it is there and otherwise the order in which motifs first appear, and dist_to_nearest holds the number of positions from each hit's middle to its nearest anchor, which is 0 inside it and negative upstream in the anchor's orientation.

Examples:

Treating each example target as a chromosome with an anchor at its middle, the W-box hits sit closer to the anchors than chance:

>>> import universalmotif as um
>>> motifs = [um.example_motifs()[i] for i in (3, 5)]
>>> targets = um.example_sequences("targets")
>>> anchors = [(name, 499, 500) for name in targets]
>>> res = um.motif_proximity(anchors=anchors, motifs=motifs, sequences=targets, nthreads=1)
>>> res.select("motif", "nhits", "best_dist", "hits_near").row(0)
('W-box', 30, 250, 27)
>>> f"{res['pvalue'][0]:.1e}"
'2.2e-05'

genomic_hits ¶

genomic_hits(hits: DataFrame, ranges=None) -> pl.DataFrame

Hits from scanning genomic ranges, moved to genomic coordinates.

scan_sequences() counts positions from the start of each sequence scanned. When those sequences are ranges taken from a genome, such as peaks read with extract_ranges(), this puts the hits back on the genome: each gets a chrom column, and start, end and strand become positions on the chromosome's forward strand, counted from 0 with the end left out. Hits in ranges that were reverse complemented have their strand flipped. The other columns are kept, so sequence still names the range each hit came from.

By default the ranges are read from the sequence names, which must be chrom:start-end with (-) after ranges on the minus strand, as extract_ranges() and match_bkg() name them. Otherwise ranges gives them in the order of the sequences scanned, as a range table or (chrom, start, end[, strand]) tuples, and hits are matched to them by sequence_i.

Hits in overlapping ranges are reported once for each, so the same site can appear more than once, and .unique(["motif", "chrom", "start", "end", "strand"], maintain_order=True) keeps one of each.

Examples:

>>> import universalmotif as um
>>> genome = {"chr1": "CCCCCTGACTCACCCCCCCCCC"}
>>> peaks = um.extract_ranges(genome, [("chr1", 2, 15), ("chr1", 2, 15, "-")])
>>> hits = um.scan_sequences(um.example_motifs()[1], peaks, pvalue=1e-3, rc=False)
>>> hits.select("sequence", "start", "end", "strand", "match").rows()
[('chr1:2-15', 2, 11, '+', 'CTGACTCAC'), ('chr1:2-15(-)', 2, 11, '+', 'GTGAGTCAG')]
>>> um.genomic_hits(hits).select("chrom", "start", "end", "strand").rows()
[('chr1', 4, 13, '+'), ('chr1', 4, 13, '-')]

to_pyranges ¶

to_pyranges(df: DataFrame)

A table of ranges, such as hits or peaks, as a PyRanges object.

The chromosome column, which can be any of chrom, seqnames, sequence, chr or chromosome, becomes Chromosome, and start, end and strand become Start, End and Strand. Positions are 0-based and half-open in both, so they are kept as they are. Other columns are kept too. PyRanges objects can be given back to any function that takes ranges.

Needs the pyranges extra.

Examples:

>>> import universalmotif as um
>>> gr = um.to_pyranges(um.genomic_hits(hits))

to_polars_bio ¶

to_polars_bio(df: DataFrame) -> pl.DataFrame

A table of ranges, such as hits or peaks, ready for polars-bio.

polars-bio works on polars DataFrames with chrom, start and end columns, and keeps their coordinate system as metadata. This renames the chromosome column to chrom if it has another name, such as the sequence of hits, moves it first, and marks the positions as 0-based, as they are throughout this package, so that polars-bio's range operations treat them correctly. polars-bio frames can be given back to any function that takes ranges.

Needs the polars-bio extra.

Examples:

>>> import universalmotif as um
>>> import polars_bio as pb
>>> hits = um.to_polars_bio(um.genomic_hits(hits))
>>> pb.overlap(hits, um.to_polars_bio(peaks))