LectureDBISAI-written · not yet proofread

VL13 Evaluation, Exam, Work in our Group

Last DBIS lecture with the official course evaluation, the exam logistics, the three exam question types with three past exam tasks, and an overview of work at the chair.

Held Tue 3 Feb 2026 Source lecture-13

Overview and Experiences with ES 12#

Overview (Slide 2):

Organizational: feedback on exercise sheet 12 (language models), last lecture of the semester.
Contents: official evaluation (results and discussion), exam info (when, where, how, what), work at the chair (how, projects, next courses).
There is no new topic and no new exercise sheet this week.

Feedback on ES 12 (Slide 3): students liked the creative tasks, and the sheet turned out easier than expected, though getting the prompt right took long. Some asked for more explanation of what is expected. One student asked how to reuse the hosted models for exam preparation if they are only hosted during that week. Slide 4 shows two “hallucinated” poems about databases from the submissions.

Course Evaluation#

Participants (Slide 5; this year only DBIS, in WS 23/24 DBIS and IR together):

	WS 25/26	WS 23/24
Registered for the exam	174	161
Participated in the evaluation	45	74

There were 29 nominations for the teaching award. The 45 participants study Computer Science (38), ESE (6) and Physics (1). All details are linked on the course Wiki.

Style of the course (Slide 6):

Question	WS 25/26	WS 23/24
Learned a lot	64% fully agree, 25% agree, 11% other	62% / 24% / 14%
Explained well	71% fully agree, 22% agree, 7% other	78% / 18% / 4%
Level of contents	46% appropriate, 46% high, 8% other	59% / 36% / 5%
Exercises	62% very good, 24% good, 13% other	57% / 33% / 10%
Quality overall	64% very good, 25% good, 11% other	67% / 26% / 7%

Those who did the exercises gave better grades.

Workload (Slide 7), effort relative to ECTS on a scale from 1 (very high) to 5 (very low):

	1	2	3	4	5
WS 25/26	9%	34%	54%	2%	0%
WS 23/24	8%	45%	45%	2%	0%
Department average	14%	26%	55%	5%	0%

The course provides the “boring” parts of the code, curated data and unit tests so that students can focus on the interesting parts. Prof. Bast still considers handling the “boring” part efficiently an important skill.

Materials, service and attendance (Slide 8):

Materials: 64% very good, 31% good, 5% okay, 3% worse (department average 27%, 37%, 22%, 14%).
Service: 58% very good, 30% good, 7% okay, 5% worse (department average 31%, 34%, 20%, 15%).
Following the lecture: mainly live 40%, mainly recordings 36%, mix 20%, mainly materials 7%. Technical support was by Frank Dal-Ri, video editing by Alexander Monneret.

Tutors and assistant (Slide 9): the tutors were Ph.D. students and postdocs of the group (Patrick Brosi, Johannes Kalmbach, Sebastian Walter, Christoph Ullinger), and the feedback on them was very positive. The assistant Robin Textor-Falconi did the technical setup, the exercise sheets and master solutions, the tutor meetings and the forum. Students are encouraged to become tutors themselves later.

Praise (Slide 10): clear structure (“one topic per lecture”), exercise sheets with project-like results, clear explanations, engaging delivery, a mix of theory and practice (“write your own engine”), helpful live coding, good materials and infrastructure, and feedback taken seriously.

Criticism (Slide 11): the most frequent comment was that the exercise sheets were too much work. Other comments: the web-app sheets were too hard, students wanted more theoretical exercises, more DBIS content instead of server and web applications at the end, and fewer slide errors.

Planned improvements (Slide 12):

More theoretical exercises. The former “Information Retrieval” course had more of them, but this is harder now because the course touches many different topics.
Make a fraction of the sheets obligatory. Fewer than 10% doing the sheets is not acceptable. The majority skip the sheets altogether, and half-hearted or plagiarized submissions are not wanted either. One idea is a system where each student commits to, say, a quarter of the sheets (spread over the semester) and does them seriously.

Lecturer’s feedback to the group (Slide 13): more people attended in person than two years ago, and those present were a pleasant crowd. On the other hand, more than 100 participants were never seen or heard from. If 80% are constantly absent, this affects the motivation of the teaching team (“reciprocity is an important thing in a society”).

Exam#

The exam overview note is Exam (not published).

Logistics#

When, where, what to bring (Slide 14):

Date: March 17, 2026 (Tuesday), 9:00 to 11:xx. Be there 15 minutes early and plan for 2.5 hours, although it might only take 2 hours.
Rooms: six rooms in building 101. The room assignment is announced in the forum.
Bring: student ID and passport (identity check), pens in two colors (improves the readability of drawings), but no red, which is reserved for the graders.

Allowed material (Slide 15):

One DIN A4 sheet, front and back, with content of your choice related to the lectures and exercises. It may be handwritten or printed, but you must create the content yourself.
No electronic devices of any kind.
There will be a sub-forum for exam questions. Responses may be slower in the semester break, so don’t leave all questions for the night before the exam.

Types of Questions#

Slide 16:

Because there were many practical exercises this year, the exam will contain more practical tasks than usual.
The content of the exercise sheets is very relevant for the exam.
Don’t learn the material schematically or by heart. Try to really understand it.

Preparation#

Slide 17:

To check understanding, put the lecture material away and write the content down or prove it in your own words or formalism. If you only memorize the individual steps of a proof, you cannot apply it in a different setting.
Once the basics are understood, the best preparation is the exercise sheets and the old exams.
For coding exercises, don’t stop when the tests pass. Make sure you understand what you are doing.
All old exams are on the Wiki.

Example Task: Ranking and Precision (WS 22/23, Task 3.1, 8 points)#

Slide 18. This task combines dot-product ranking in the vector space model with P@k, P@R and average precision from the Evaluation topic.

SuccessSolution

Following the official solution (exam WS 22/23 (not published)).

For $q = (0, 1, 3)^T$ , $q^T A = (7, 4, 6, 5, 3)$ , so the scores are $D_1 = 7 > D_3 = 6 > D_4 = 5 > D_2 = 4 > D_5 = 3$ , which is the required ranking.
Let $R$ be the number of relevant documents. $P@R = 0.5$ means that exactly $R/2$ of the top $R$ are relevant, so $R$ is even. $R = 4$ would give $P@4 = 0.5 \neq 0.25$ , so $R = 2$ . Then exactly one of the top 2 is relevant, no further one is in ranks 3 and 4 (since $P@4 = 1/4$ ), and the second relevant document is $D_5$ at rank 5. The official solution picks $\{D_3, D_5\}$ .
$\text{AP} = \frac{1}{R} \sum_{k:\ \text{rank } k \text{ relevant}} P@k = \frac{1}{2}\left(P@2 + P@5\right) = \frac{1}{2}\left(\frac{1}{2} + \frac{2}{5}\right) = 0.45$ .

Example Task: Query Plan Costs (WS 23/24, Task 2.2, 8 points)#

Slide 19. This task uses the cost estimates of the Relational Algebra operations from SQL and Query Planning.

QuestionTask

Compute the cost of each of these two query plans, where $A$ has shape $10 \times 4$ and $B$ has shape $20 \times 2$ (the exam provides the cost estimate functions). For which shapes of $A$ and $B$ is plan 2 cheaper than plan 1?

Query plan 1                       Query plan 2
1. A' = select(A, Φ1)              1. C   = cart-prod(A, B)
2. B' = select(B, Φ2)              2. C'  = select(C, Φ1)
3. C  = cart-prod(A', B')          3. C'' = select(C', Φ2)
4. R  = project(C, all cols)       4. R   = project(C'', all cols)

SuccessSolution

The exam’s cost model (exam WS 23/24 (not published)): for a table with $n$ rows and $k$ columns, select costs $n \cdot k$ and project costs $n \cdot k'$ ( $k'$ = number of kept columns). The cartesian product of $T_1$ ( $n_1 \times k_1$ ) and $T_2$ ( $n_2 \times k_2$ ) costs $n_1 n_2 (k_1 + k_2)$ . Each select halves the number of rows.

Step	Plan 1	Output	Plan 2	Output
1	$10 \cdot 4 = 40$	$5 \times 4$	$10 \cdot 20 \cdot 6 = 1200$	$200 \times 6$
2	$20 \cdot 2 = 40$	$10 \times 2$	$200 \cdot 6 = 1200$	$100 \times 6$
3	$5 \cdot 10 \cdot 6 = 300$	$50 \times 6$	$100 \cdot 6 = 600$	$50 \times 6$
4	$50 \cdot 6 = 300$	$50 \times 6$	$50 \cdot 6 = 300$	$50 \times 6$
Total	680		3300

In general, for $A$ of shape $n_A \times k_A$ and $B$ of shape $n_B \times k_B$ , let $P = n_A n_B (k_A + k_B)$ . Then

\text{cost}_1 = n_A k_A + n_B k_B + \tfrac{P}{4} + \tfrac{P}{4}, \qquad \text{cost}_2 = P + P + \tfrac{P}{2} + \tfrac{P}{4},

so plan 2 is cheaper iff $\tfrac{9}{4} P < n_A k_A + n_B k_B$ . For $n_A, n_B \geq 1$ we have $P \geq n_A k_A + n_B k_B$ , so this never holds. Plan 2 is cheaper only if one table is empty (e.g. $n_A = 0$ , $n_B > 0$ ). Then the product costs 0, while plan 1 still pays for selecting the non-empty table.

Example Task: Q-Grams and Edit Distance (WS 22/23, Task 2.2, 5 points)#

Slide 20. This task applies the Q-Gram Lemma from Fuzzy Search and Q-Grams.

SuccessSolution

Following the official solution (exam WS 22/23 (not published)). Let $x, y$ be the strings and $x', y'$ the versions padded with $q - 1 = 2$ $ on both sides.

$|x'| = |y'| = 5 + 2 \cdot 2 = 9$ , so each has $9 - 3 + 1 = 7$ 3-grams: $|Q_3(x')| = |Q_3(y')| = 7$ .
Q-gram lemma: $\text{comm}_3(x', y') \geq \max\{|Q_3(x')|, |Q_3(y')|\} - q \cdot \text{ED}(x, y)$ , so $3 \geq 7 - 3 \cdot \text{ED}(x, y)$ .
Hence $\text{ED}(x, y) \geq 4/3$ , and since ED is an integer, $\text{ED}(x, y) \geq 2$ . The minimal possible edit distance is 2.

Work at the Chair#

How the group works (Slide 21):

It solves practically relevant, non-trivial problems in search, knowledge graphs, natural language processing, geographic information systems and related areas, and has many large and interesting datasets.
It builds software that is useful and actually used. This requires good code, documentation and user interfaces.
It uses theory mostly as a tool, but an essential one. Without a solid theoretical basis, solving complex problems stays hacking and guesswork.
All three aspects were part of this course too. The only difference is that research projects use git instead of svn.

Supervision (Slide 22): very good infrastructure and support, and a lot of freedom in what and how you work. That freedom suits some people well and others less, since some prefer step-by-step instructions. The group fits people who are happy when the result is useful and used, and who like to solve problems, code and get things done. The problems don’t have to be extremely hard. Usefulness, insight or aesthetics matter more.

Artificial intelligence (Slide 23, Slide 24):

The group uses AI in many places, especially for natural language processing. This basic course contains little of it. The advanced course in WS 26/27 covers latent semantic indexing, multinomial regression, large language models in depth, question answering and more.
The group also still does a lot of classical algorithmics. Tuning complex neural networks until they work is not always fun.
AI groups in Freiburg: Hutter (hyperparameter optimization, AutoML), Brox (computer vision), Valada (robotics), Bödecker (neurorobotics), Kalweit (AI and medicine), Kamingsun (foundations of AI), Bast (natural language processing).

Current projects with demos (Slide 25):

Travic: worldwide public transit visualization.
QLever: the open-source RDF database.
Petrimaps: visualization of large geo results.
QLever UI++: a new user interface.
GRASP: question answering.

Upcoming courses (Slide 26):

Programmieren in C++ (SS 2026): basic course for all B.Sc. students of the faculty. Tutors wanted.
DBIS, advanced course (WS 26/27): advanced knowledge graphs, compression, efficient list intersection, and the AI topics from slide 23.
B.Sc./M.Sc. projects and theses: offered all the time. Read the instructions on the Wiki and then ask, and ask again if there is no response. Showing commitment in one of the group’s lectures helps.

Concepts#

Vector Space Model (VSM) — dot-product ranking $q^T A$ over a term-document matrix in the WS 22/23 Task 3.1 example
Precision at k — P@4 and P@R conditions that determine the relevant documents in Task 3.1
Average Precision — computed for the chosen relevant set in Task 3.1
Relational Algebra — cost estimates of select, project and cartesian product for the query plan comparison in WS 23/24 Task 2.2
Edit Distance — the quantity bounded from below in WS 22/23 Task 2.2
Q-Gram — padded 3-grams of length-5 strings in WS 22/23 Task 2.2
Q-Gram Lemma — gives the lower bound $\text{ED} \geq 2$ in WS 22/23 Task 2.2
Exam question types — the three categories (basic understanding, coding, deeper understanding) of the DBIS exam