2025-05-31-pycon_italia

Keynote given at PyCon Italia 2025: https://2025.pycon.it/en/keynotes/data-doesnt-lie-but-it-can-mislead-how-to-ensure-integrity-of-your-machine-learning-applications

Video available here: https://youtu.be/qXx-4m-Z0q4?t=29341 (8:09:01 - 8:55:25)

Transcript

1. Data doesn't lie, but it can mislead How to ensure integrity of your ML applications
2. Me: Sofie Van Landeghem Open-source contributor Freelance NLP consultant https://oxykodit.com/
3. Create a portal with all Formula One news
4. Date Race Winner 4 May 2025 Miami Oscar Piastri 18 May 2025 Emilia Romagna Max Verstappen 25 May 2025 Monaco Lando Norris Link facts to given news articles
5. Date Race Winner 28 July 2024 Spa George Russell Let's inspect some sample data
6. But in reality…
7. Date Race Winner 28 July 2024 Spa Lewis Hamilton Date Race Winner 28 July 2024 Spa George Russell 28 July 2024 Spa Lewis Hamilton NLP Filtering
8. Episode "Looking Out for Number 1" A narrative about George Russell "stepping up" into a leadership role at Mercedes … narrating the Spa 2024 Grand Prix … … showing Russell on the podium … … and never even mentioning his disqualification.
10. Always have a domain expert on board who can double check the data and the results
11. Portal requirement: link-out to Wikipedia
12. "Hamilton won the Belgian Grand Prix in Spa in 2024."
13. Obtaining gold-standard data Sir Lewis Carl Davidson Hamilton (born 7 January 1985) is a British [[racing driver]] who competes in [[Formula One]] for [[Scuderia Ferrari|Ferrari]].
14. Building an Entity Linker model
15. ✦ 79% F-score ✦
16. 79% is pretty good, right? Right?
17. How most clients / users think about performance 0% 50% 100% The worst model "Random guessing" The best model "Entirely correct"
18. … but a random baseline is almost never 50%
19. For our entity linking task, the random baseline is actually 1/6.997.326 = 0,000014% (a.k.a. 0%)
20. … and our upper bound wasn't 100% either …
21. In fact, we had to prune the database for efficiency requirements ❖ Kept only 14% of all Wikipedia concepts ❖ An "oracle" disambiguation obtains 84% F-score So, the modeling work targets a 0% - 84% range
22. Ok, so within a range of 0-84%, 79% is really good, right? Right?
23. Develop some simple baselines to better understand the complexity and data challenges of your project
24. In our case, we won't assign "Hamilton" to a random page out of the 7M available ones
25. Instead, we obtain a list of "Hamilton" candidates through lexical search
26. Now, if we pick a random one from this candidate list, we actually obtain 54% F-score (without doing any ML at all!)
27. Let's create an even stronger baseline.
28. Consider "prior probabilities": a measure of how often a mention is linked to a certain concept Textual mention F1 race driver 18th century military officer Duke of Hamilton "Hamilton" 75% 7% 10% "Lewis Hamilton" 99% 0% 0% "Alexander Hamilton" 0% 97% 1%
29. Now extrapolate them to the extreme and use this as "predictions": Textual mention F1 race driver 18th century military officer Duke of Hamilton "Hamilton" 100% 0% 0% "Lewis Hamilton" 100% 0% 0% "Alexander Hamilton" 0% 100% 0%
30. This "prior probabilities" baseline obtains 78.2% F-score (still without any ML at all!)
31. The ML model that is supposed to disambiguate based on context (79%) only marginally improves upon a relatively simple baseline (78.2%)
32. Developing a few simple baselines puts your ML's performance measures in the right perspective
33. Let's revisit our upper bound (again). Is it really 100% if we have no memory / efficiency requirements? 0 100% The best model "Entirely correct"
34. 100% precision means every disambiguation is "correct". But what does "correctness" even mean?
35. "Hamilton won the Belgian Grand Prix in Spa in 2024." ? ?
36. "Societies in the ancient civilizations of Greece and Rome preferred small families" ? ?
37. "Full metro systems are in operation in Paris, Lyon and Marseille" ? ?
38. When multiple annotators label the same data sample, how often do they agree? ↪ Inter-annotator agreement = IAA
39. Annotators mostly agree → the gold data is reliable and robust Annotators disagree because… ★ The data is confusing ★ The label scheme is ambiguous ★ The NLP task is too complex High IAA Low IAA
40. We need to label incoming articles for our portal 1. Sports a. Football b. F1 2. Politics a. Global politics b. US politics 3. Leisure a. Traveling b. Sports c. Literature
41. Let's plot IAA of the labels… Football F1 Global politics US Politics Traveling Sports Literature Football 0.79 F1 0 0.73 Global politics 0 0 0.95 US Politics 0 0.12 0.05 0.83 Traveling 0 0 0 0 1.0 Sports 0.21 0.15 0 0 0 0.64 Literature 0 0 0 0 0 0 1.0
42. Confusion between global and US politics? Football F1 Global politics US Politics Traveling Sports Literature Football 0.79 F1 0 0.73 Global politics 0 0 0.95 US Politics 0 0.12 0.05 0.83 Traveling 0 0 0 0 1.0 Sports 0.21 0.15 0 0 0 0.64 Literature 0 0 0 0 0 0 1.0
43. Consider merging labels if humans can't reliably distinguish them
44. Confusion between politics and F1? Football F1 Global politics US Politics Traveling Sports Literature Football 0.79 F1 0 0.73 Global politics 0 0 0.95 US Politics 0 0.12 0.05 0.83 Traveling 0 0 0 0 1.0 Sports 0.21 0.15 0 0 0 0.64 Literature 0 0 0 0 0 0 1.0
45. Solution: allow multiple correct labels 1. Sports a. Football b. F1 2. Politics a. Global politics b. US politics 3. Leisure a. Traveling b. Sports c. Literature
46. Reframe the task to match your data
47. Confusion within the sports category? Football F1 Global politics US Politics Traveling Sports Literature Football 0.79 F1 0 0.73 Global politics 0 0 0.95 US Politics 0 0.12 0.05 0.83 Traveling 0 0 0 0 1.0 Sports 0.21 0.15 0 0 0 0.64 Literature 0 0 0 0 0 0 1.0
48. 1. Sports a. Football b. F1 2. Politics a. Global politics b. US politics 3. Leisure a. Traveling b. Sports c. Literature Solution: critically revise the label scheme given to you! Inherent ambiguity in the label scheme needs to be addressed
49. Together with your domain experts, critically revise the label scheme
50. What if we don't have multiple annotators?
51. Analyse discrepancies between a model's prediction and the gold standard label
52. "Agnes Maria of Andechs-Merania (died 1201) was a Queen of France." Gold annotation Prediction
53. Prediction Gold ≠ Is this really an incorrect prediction (a "false positive")? Or is the label wrongly annotated? Perhaps both answers can be seen as "correct"?
54. Depending on the downstream use-case, the precision of this method was 87-96%
55. Analyse "wrong" predictions on the training dataset, to find structural data errors
56. Manually annotated training dataset: "... to kick off 2025 [Dutch Grand Prix] weekend in Zandvoort" "Gasly laments ‘quite sad’ [Monaco] GP crash" Sample prediction: "Gasly laments ‘quite sad’ [Monaco GP] crash"
57. Always include the words "Grand Prix" or "GP" into the entity annotation: ❌ Gasly laments ‘quite sad’ [Monaco] GP crash ✅ Gasly laments ‘quite sad’ [Monaco GP] crash Write up annotation guidelines to help with consistency of annotations
58. "Correctness" may also depend on the downstream usage of your results
59. "Hamilton won the Belgian Grand Prix in Spa in 2024." ? ?
60. "Hamilton won the Belgian Grand Prix in Spa in 2024." ? ?
61. Make sure you climb the right hill
62. Is your data correct, consistent and robust? Is your test data representative of real-world data? Does your evaluation capture the value of the algorithm in production?
63. Only after defining the right hill to climb, you can get to the fun stuff: ★ Algorithm development ★ Model training ★ LLM fine-tuning ★ …
64. Let's wrap up with a practical checklist 📝
65. ML project: data annotation ✓ Ensure your label scheme is consistent and unambiguous ✓ Draft clear annotation guidelines to ensure data consistency ✓ Measure inter-annotator agreement (IAA) ✓ Consider reframing your task/guidelines if the IAA is low ✓ Model uncertainty in your annotation workflow 📝 1/3
66. ML project: performance evaluation ✓ Develop simple baselines to put performance into perspective ✓ Quantify realistic upper/lower performance bounds ✓ Measure performance as part of the larger business process 📝 2/3
67. ML project: performance evaluation ✓ Identify structural data errors by "predicting" the training data ✓ Apply to truly unseen data to measure realistic performance ✓ Make sure you’re climbing the right hill 📝 3/3
68. Main Take Away Hamilton won the Belgian 2024 Grand Prix, not Russell ;-)
69. Thank you! Sofie Van Landeghem NLP freelancer @ OxyKodit PyCon Italia 2025