ConceptDBISAI-written · not yet proofread

Mean Average Precision

Also MAP · MP@k · MP@R

Mean average precision (MAP) is the arithmetic mean of the average precision AP(q) over all queries q of a benchmark.

Formula#

The same averaging over a benchmark applies to every per-query measure; the lecture names three (DBIS_VL02 > Mean Precisions over a Benchmark):

with $\mathrm{AP}(q) = \frac{1}{R_q} \sum_{k:\, r_k \text{ relevant}} \mathrm{P@}k(q)$ and $R_q$ the number of relevant documents for $q$ .

Single-query measures (P@k, AP) rate one ranking; MAP and MP@k/MP@R rate a system on a whole benchmark. They are very common in research papers on ranking.
$\mathrm{MAP} \in [0, 1]$ ; $\mathrm{MAP} = 1$ iff $\mathrm{AP}(q) = 1$ for every query, i.e. for every query all relevant documents come before all non-relevant ones.
Every query has the same weight $1/|Q|$ , regardless of how many relevant documents it has.
Like AP, MAP assumes that all relevant documents are known; with incomplete judgments use bpref.
Tuning parameters to maximize MAP on the benchmark used for reporting inflates it, see Overfitting (IR Evaluation).

Connections

Average Precision
Averaged over all queries of a benchmark it gives MAP. §
Ground Truth
Every evaluation measure compares a ranked result list with the ground truth: P@k, P@R, AP, MAP, nDCG and bpref. $R = |$ ground truth of $q|$ is the number of relevant documents. §
Overfitting (IR Evaluation)
Let $M(\theta, Q)$ be a quality measure (e.g. MAP) of a system with parameters $\theta$ (e.g. the BM25 parameters $k, b$ ) on the query set $Q$ . Choosing §
Precision at k
Averaged over all queries of a benchmark: $\mathrm{MP@}k$ and $\mathrm{MP@R}$ , see Mean Average Precision. §
VL02 – Ranking and Evaluation
MAP = mean of the AP values over all queries. §
MAP — mean AP over a benchmark (with MP@k, MP@R) §
Evaluation
Mean Average Precision — mean of AP (and P@k, P@R) over a benchmark §
Per query vs. per system. P@k, AP, nDCG, bpref rate one result list; averaging over all queries of the benchmark gives MP@k, MP@R, MAP (Mean Average Precision), which rate the system. The ground truth is a set but the results a ranked list… §