Why are the results of AUCell not centered around 0.5 (for a random "regulon")?

[scyrusm]

Hi there,

I'm trying to better understand how to interpret the output of AUCell. In particular, if we were to calculate the ranking of genes in a regulon, and plot on the x axis the ranking, and the y axis the cumulative number of genes in the regulon at or above the corresponding ranking, we would have a typical AUC/ROC curve. A "regulon" consisting of randomly selected genes would have an AUC of 0.5, the maximally enriched regulon having and AUC of 1, and the maximally suppressed having one bounded below by 0. This is equivalent (ignoring the case of ties) to the common language effect size between the rankings of the genes in the regulon and the genes not in the regulon, on a cell-by-cell basis.

I suspect that this is due to the parameters --rank_threshold and --auc_threshold, but I am not sure.

In practice, the output of AUCell seems to mostly be between 0 and 0.5, including for genes that seem to be "enriched." And, from what I can tell, downstream analysis seems to suggest using the AUC more as a summary statistic (for example, by using mixture models to binarize the distribution of the AUC).

It seems that this doesn't agree with the more conventional definition of the AUC/ROC (here). Why was this chosen? And, given that, how do we interpret the outputs of pyscenic aucell?

EDIT:
As an added note, the following code from ctxcore leads to the perverse situation where setting auc_threshold to 1 will lead to an assertion error: if auc_threshold were 1, rank_cutoff will be equal to total_genes, but rank_threshold has been set to total_genes - 1. So rank_threshold can at most be total_genes - 1, but then in the final line, rank_cutoff is decremented by `. It seems that there was a double attempt to fix the 0- vs 1-indexing discrepancy between python and R, leading to an off-by-one error...

def derive_rank_cutoff(
auc_threshold: float, total_genes: int, rank_threshold: Optional[int] = None
) -> int:
"""
Get rank cutoff.

:param auc_threshold: The fraction of the ranked genome to take into account for
    the calculation of the Area Under the recovery Curve.
:param total_genes: The total number of genes ranked.
:param rank_threshold: The total number of ranked genes to take into account when
    creating a recovery curve.
:return Rank cutoff.
"""

if not rank_threshold:
    rank_threshold = total_genes - 1

assert (
    0 < rank_threshold < total_genes
), f"Rank threshold must be an integer between 1 and {total_genes:d}."
assert (
    0.0 < auc_threshold <= 1.0
), "AUC threshold must be a fraction between 0.0 and 1.0."

# In the R implementation the cutoff is rounded.
rank_cutoff = int(round(auc_threshold * total_genes))
assert 0 < rank_cutoff <= rank_threshold, (
    f"An AUC threshold of {auc_threshold:f} corresponds to {rank_cutoff:d} top "                                                                                                                               
    f"ranked genes/regions in the database. Please increase the rank threshold "                                                                                                                               
    "or decrease the AUC threshold."                                                                                                                                                                           
)                                                                                                                                                                                                              
# Make sure we have exactly the same AUC values as the R-SCENIC pipeline.                                                                                                                                      
# In the latter the rank threshold is not included in AUC calculation.                                                                                                                                         
rank_cutoff -= 1
return rank_cutoff