CA Part 2 Grading
Required files (place in your repo)
- ✅
.github/workflows/classroom.yml
- ✅
tools/evaluate_final.py
- Put them exactly in
.github/workflows/ and tools/ respectively. Do not edit these files.
What the workflow does
- On push/CI,
.github/workflows/classroom.yml runs.
- It downloads assets on the runner (topics/qrels/dataset), builds your code, runs search to produce
runs/student.run, evaluates with tools/evaluate_final.py, and submits metrics. Only minimal status is shown.
How to submit
- Add the provided files to the correct paths (
.github/workflows/classroom.yml, tools/evaluate_final.py).
- Ensure your code can build and writes
runs/student.run in TREC format (local sanity check optional).
- Commit only your retrieval code/configs; leave workflow/eval scripts untouched.
- Push your repo; CI runs automatically.
- If CI fails, make minimal fixes to your own code and push again. Explain the fixes to the code in your commit message precisely.
What you should expect after submission
- In GitHub Actions: the workflow completes with green checks; you only see minimal status.
- Scores/metrics are submitted to the final grades system; detailed metrics are not shown to you until published.
Rules
- Do not modify
classroom.yml or tools/evaluate_final.py.
- Do not change your retrieval/search implementation or any submitted code unless explicitly told to resubmit.
- Keep repo structure unchanged;
runs/student.run must be produced in TREC format.
- No local downloads of topics/qrels/dataset are needed—handled on the runner.
- Do not alter the submitted repository after pushing (unless instructed to resubmit).
Contact