ConceptDBISAI-written · not yet proofread

Precision at k

Also P@k · P@R · Precision at R

P@k is the fraction of relevant documents among the top k results of a ranked result list; P@R is P@k with k = R, the number of relevant documents for the query.

Know these firstGround Truth

Properties#

$\mathrm{P@}k \in [0, 1]$ ; $\mathrm{P@}k = 1$ iff all top- $k$ results are relevant.
$\mathrm{P@}k$ ignores the order within the top $k$ and everything below rank $k$ ; it does not depend on $R$ . For $k > R$ , $\mathrm{P@}k \le R/k < 1$ even for a perfect ranking.
$\mathrm{P@R}$ adapts the cut-off to the query: $\mathrm{P@R} = 1$ iff the first $R$ results are exactly the $R$ relevant documents.
$\mathrm{P@}k$ is not monotone in $k$ (see the example: $1, 0.5, 0.33, 0.5, 0.6$ ).
Averaged over all queries of a benchmark: $\mathrm{MP@}k$ and $\mathrm{MP@R}$ , see Mean Average Precision.
Average Precision averages $\mathrm{P@}k$ over the positions $k$ of the relevant documents.

Example#

ExampleQuery matrix movies (lecture)

Relevant: 10, 582, 877, 10003 ( $R = 4$ ). Result list: 582 (rel), 17, 5666, 10003 (rel), 10 (rel), 37, …

$k$	1	2	3	4	5
$\mathrm{P@}k$	$1/1 = 1$	$1/2 = 0.5$	$1/3 \approx 0.33$	$2/4 = 0.5$	$3/5 = 0.6$

$\mathrm{P@R} = \mathrm{P@4} = 0.5$ .

ExampleExam WS 23/24, Task 1.3

Result $[6, 7, 2, 4, 5]$ , relevant $\{1, 2, 3, 4\}$ , so $R = 4$ and the relevance by position is $[0, 0, 1, 1, 0]$ .

Metric	Calculation	Value
$\mathrm{P@2}$	$0/2$	$0$
$\mathrm{P@3}$	$1/3$	$1/3$
$\mathrm{P@R} = \mathrm{P@4}$	$2/4$	$1/2$

Sources#

Slides 02, p. 22, Slides 02, p. 24 (MP@k, MP@R), Slides 13, p. 18
DBIS_VL02 > Precision at k, DBIS_VL13 > Example Task: Ranking and Precision (WS 22/23, Task 3.1, 8 points)
Exams: DBIS Exam WS1617 (not published) Task 1.3, DBIS Exam WS1718 (not published) Task 1.1, DBIS Exam WS2122 (not published) Task 1.3, DBIS Exam WS2223 (not published) Task 3.1, DBIS Exam WS2324 (not published) Task 1.3

Connections

Mentioned in inline links7

Average Precision
Let a query have $R$ relevant documents (ground truth), and let $R_1 < R_2 < \dots < R_R$ be their sorted positions in the result list. The average precision is the average of the $R$ values $\mathrm{P@}R_i$ (Precision at k at the position… §
Discounted Cumulative Gain
Discount: the factor $1/\log_2(i+1)$ makes a relevant document count less the lower it is ranked; rank 1 keeps its full relevance ( $\log_2 2 = 1$ ). So very relevant documents ranked above only somewhat relevant ones get a bonus, which bina… §
Ground Truth
Every evaluation measure compares a ranked result list with the ground truth: P@k, P@R, AP, MAP, nDCG and bpref. $R = |$ ground truth of $q|$ is the number of relevant documents. §
Mean Average Precision
Single-query measures (P@k, AP) rate one ranking; MAP and MP@k/MP@R rate a system on a whole benchmark. They are very common in research papers on ranking. §
VL02 – Ranking and Evaluation
For a result list of a query, $\mathrm{P@}k$ is the fraction of relevant documents among the top $k$ results. $\mathrm{P@R} = \mathrm{P@}k$ with $k = R$ the number of relevant documents for the query. (Slide 22, Precision at k) §
Precision at k — P@k and P@R, fraction of relevant hits in the top $k$ §
VL13 – Evaluation, Exam, Work in our Group
Slide 18. This task combines dot-product ranking in the vector space model with P@k, P@R and average precision from the Evaluation topic. §
Precision at k — P@4 and P@R conditions that determine the relevant documents in Task 3.1 §
Evaluation
Precision at k — P@k and P@R, fraction of relevant hits in the top $k$ / top $R$ §
| binary, complete | P@k | top $k$ only | ignores order in the top $k$ and everything below | §

Search in these mentions →