Data Analysis with Kotlin Notebook, DataFrame, and Kandy

Transcript

1. Data Analysis with Kotlin Notebook @antonarhipov
3. De fi ne objective
4. De fi ne objective Collect data
5. De fi ne objective Collect data Clean the data
6. De fi ne objective Collect data Clean data Explore data
7. De fi ne objective Collect data Clean data Explore data Analyze data
8. De fi ne objective Collect data Clean data Explore data Analyze data Interpret the results Communicate fi ndings Implement decisions Monitor decisions
9. De fi Collect data Clean data Analyze data Interpret the results Communicate fi Implement decisions Monitor decisions Exploratory Data Anasysis (EDA) Explore data This is what we are focusing on in this presentation
10. What does it take to hit the charts? spotify-2023.csv https://www.kaggle.com/datasets/nelgiriyewithana/top-spotify-songs-2023
13. Read data from CSV fi le and create a DataFrame instance
14. Expose the variable from the cell
15. Get a glimpse into the schema
16. Looks like Taylor Swift is the most popular artist with 34 appearances
17. Nullable type indicates that the data ins't always present
18. The streams count is identi fi ed as String. This is clearly wrong in this case!
19. Aha! An invalid data entry!
20. This song isn't present in any charts. Looks like we can just ignore it
21. Adjust the structure for convenience
22. Filter out the invalid data
23. Replace column data type
24. Move the column to the left for convenience
25. Rename columns
26. Sort the data set
28. Most popular artists by appearance
29. Use Kandy library to visualize the data
30. Top 20 artists Plot is an extension function for DataFrame Map the data to axes Add some colors
32. Looks like you just need to be Taylor Swift to hit the charts ;)
33. Not in Top 20?
34. Not in Top 30?
35. Not even in Top 50?
36. OK, 1 appearance in Top 100
37. Let's take a look at the top songs
38. New column merging the values from the 'title' and the 'artist'
39. Evenly distribute the values between the color categories
40. Looks like C# key is the most popular one
41. Let's try fi nding correlations between the di ff erent attributes related to the streams count
42. Looks like there are some attributes that correlate better than the others
43. "Gather" - prepare the data for plotting
44. A bit brighter spots are the points of interest
45. A bit brighter spots are the points of interest The outcome: Danceability is useful And you need to have at least some energy levels to be popular
46. Analyzing the BPM (tempo) of the song
47. Aggregation operation creates a new column that will be statically accessible in the next cell
48. The static accessor was generated for the new column created by the aggregate operation
49. The static accessor was generated for the new column created by the aggregate operation It looks like the most popular songs are contained within 90 to 130 BPM range The overall median is 121 BPM The most popular BPM is 120 - 39 occurances
50. C# is the most popular key among the top songs on Spotify
51. Some (very naive) outcomes: BPM matters: not too slow, not too fast, 120 is good The most popular key is C# Is probably G, G#, and D is acceptable too Having at least some energy levels in the song is useful
52. Resources: Kotlin Notebook plugin Kotlin DataFrame Kotlin Kandy https://plugins.jetbrains.com/plugin/16340-kotlin-notebook https://kotlin.github.io/dataframe https://kotlin.github.io/kandy Demo https://github.com/antonarhipov/kotlin-notebooks-demo/
53. https://speakerdeck.com/antonarhipov @antonarhipov https://github.com/antonarhipov