ConceptDBISProofread

Vector Space Model

Also VSM

In the vector space model each document is a vector of term scores (a column of the term-document matrix A) and a query is a 0/1 term vector q, so the relevance scores of all documents are the dot products q^T A.

Documents are represented as vectors. A vector describes what words appear how often in a given document (like term frequencies)

words are terms
term-document matrix: All vectors side by side

Retrieval#

represent the query as vector
measure relevancy of each document to the query by taking the dot product
- This again corresponse to the term frequencies from Lecture 2 Slide 10
- This can be done for each document with one matrix multiplication

Another method to measure relevancy is using the cos-similarity. This essentially normalizes for the length, the dot product measures the “alignment”.

Effiecient storing#

In reality, most of the entries will be $0$ . Therefor a good model to store the matix is with “sparce matrices”

-> Store the non-zero entries together with its location

Example#

TodoSuggested by Claude

## Example is empty. Candidate: the lecture’s running example (terms internet, web, surfing, beach; documents D1–D6) with the query “web surfing”, $q = (0, 1, 1, 0)^T$ , $q^T A = (2, 1, 2, 3, 1, 1)$ , so D4 ranks first (DBIS_VL11 > Retrieval as a matrix-vector product).
The dot product with tf entries reproduces the tf score sum of Lecture 2; with BM25 entries it gives exactly the BM25 ranking of the inverted index (Slides 11, p. 11). Worth a line under Retrieval.
Typos: “corresponse” → “corresponds”, “Effiecient” → “Efficient”, “Therefor” → “Therefore”, “sparce” → “sparse” (link sparse matrices), “matix” → “matrix”.
Structure (spec §5.3): no # Vector Space Model title and no [!definition] callout; # Retrieval and # Effiecient storing are H1 and would become ##. The slide: key would become a ## Sources bullet.

Connections

Mentioned in inline links11

Embedding
Sum = bag of words: the summed text embedding, like the VSM document vector (sum of one-hot encodings), ignores word order because addition is commutative: “cute cats eating” and “eating cute cats” get the same vector and cos-sim $1$ . §
Inverted Index
In practice, most entries are zero (sparse), so the inverted index is far more compact than a full term-document matrix. §
LSI Ranking
Let $A \in \mathbb{R}^{m \times n}$ be the term-document matrix of the Vector Space Model ( $m$ terms, $n$ documents, e.g. tf.idf entries) and $A = U S V$ its reduced SVD with $U \in \mathbb{R}^{m \times r}$ , $S \in \mathbb{R}^{r \times r}$ … §
One-Hot Encoding
Documents and queries are sums: a document column of the term-document matrix in the Vector Space Model (VSM) is the sum of the one-hot encodings of its words, with multiplicity: $d = \sum_{w \in \text{doc}} e_w$ (with tf entries). A query… §
Sparse Matrix
Typical case: a term-document matrix in the Vector Space Model, where $|V|$ is large but each document contains only a small fraction of the vocabulary, so storing all entries is infeasible. §
TF-IDF
Arranging the tf.idf scores of all terms and documents gives the term-document matrix $A$ of the Vector Space Model, with rows = terms, columns = documents: §
Term Frequency
In the Vector Space Model with the tf term-document matrix $A$ ( $A_{ij} = \mathrm{tf}(t_i, D_j)$ ) and the 0/1 query vector $q$ , this is $q^T A$ for all documents at once. §
VL11 – Vector Space Model, Word Embeddings
In the Vector Space Model (VSM) each document is a vector with one component per word; in this context words are called terms. Putting the vectors of all documents of a corpus side by side gives the term-document matrix $A$ (rows = terms,… §
Vector Space Model (VSM) — documents as columns of a term-document matrix, retrieval as $q^T A$ §
VL13 – Evaluation, Exam, Work in our Group
Slide 18. This task combines dot-product ranking in the vector space model with P@k, P@R and average precision from the Evaluation topic. §
Vector Space Model (VSM) — dot-product ranking $q^T A$ over a term-document matrix in the WS 22/23 Task 3.1 example §
Latent Semantic Indexing
Starting point. In the vector space model the term-document matrix $A \in \mathbb{R}^{m \times n}$ ( $m$ terms, $n$ documents) scores all documents for a query vector $q \in \mathbb{R}^m$ at once as $q^\top A$ . A document only gets a score… §
Word Embeddings and Language Models
Vector Space Model — documents as term-score vectors in a term-document matrix $A$ ; retrieval as $q^\top A$ §
| VSM / one-hot | $e_i$ , orthogonal | sum of one-hot encodings = column of $A$ (tf), or BM25 scores | $\lvert V \rvert$ , sparse | cos-sim $0$ | lost (bag of words) | §

Search in these mentions →