Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Uladzislau Sazanovich - JetBrains AI: Deep Dive

Uladzislau Sazanovich - JetBrains AI: Deep Dive

A deep dive into the architecture and implementation of JetBrains AI – the backbone of AI Assistant in JetBrains IDEs, Grazie, and other JetBrains products. The JetBrains AI platform covers a broad range of functionality, from coding to writing assistance. We will discuss both JetBrains’s own models and the caveats of integration with some well-known LLM providers. We will also shed light on inference techniques used in JetBrains AI products.

MunichDataGeeks

December 05, 2023
Tweet

More Decks by MunichDataGeeks

Other Decks in Technology

Transcript

  1. fun testGeneration() { val lm = LanguageModel() val prefix =

    "Once upon a time " Next word prediction, using prefix 12
  2. fun testGeneration() { val lm = LanguageModel() val prefix =

    "Once upon a time " Language modeling! 13 Next word prediction, using prefix
  3. fun testGeneration() { val lm = LanguageModel() val prefix =

    "Once upon a time " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } 15
  4. fun testGeneration() { val lm = LanguageModel() val prefix =

    "Once upon a time " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } Doesn’t make any sense 16
  5. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } 17
  6. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } Prefix 18
  7. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } Suffix Prefix 19
  8. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } file: src/test/com/dogs/retriever/Generate.kt 20
  9. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } file: src/test/com/dogs/retriever/Generate.kt Suffix Prefix One-line completion 21
  10. 22

  11. Multi-line completion ./ Example of LanguageModel usage fun runGeneration() {

    val lm = LanguageModel() val prefix = "My dog is " val generation = lm.predict(prefix) } 24
  12. Multi-line completion ./ Example of LanguageModel usage fun runGeneration() {

    val lm = LanguageModel() val prefix = "My dog is " val generation = lm.predict(prefix) } Much harder task 25
  13. 26

  14. Prompt We want all of it 28 Type Declarations Local

    Variables Base Class Relevant Files MORE!
  15. IDE Multi-line context fun runGeneration() { val lm = LanguageModel()

    val prefix = "My dog is " val generation = lm.predict(prefix) } 33
  16. IDE LanguageModel.kt class LanguageModel() { • fun predict(prefix: String) {

    ... } } Multi-line context fun runGeneration() { val lm = LanguageModel() val prefix = "My dog is " val generation = lm.predict(prefix) } 34
  17. IDE LanguageModel.kt class LanguageModel() { • fun predict(prefix: String) {

    ... } } Multi-line context fun runGeneration() { val lm = LanguageModel() val prefix = "My dog is " val generation = lm.predict(prefix) } ProjectInfo file: Run.kt lang: Kotlin 1.9 libraries: ... 35
  18. 36

  19. fun testGeneration() { val lm = LanguageModel() val prefix =

    "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } Just one line 41
  20. LLMs are great • Intelligent • Huge context • Easy

    to prototype • Slow and expensive 43
  21. val prefix = "My dog is golden " val expected

    = "retriever" val prefix = "My dog is golden " val exp = 1e6 MATCH MISS :( Line Accuracy 52
  22. 56

  23. class LanguageModel { fun predict(prefix: String) { ... } }

    fun testGeneration() { val lm = LanguageModel() val prefix = "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } 66
  24. class LanguageModel { fun predict(prefix: String) { ... } }

    fun testGeneration() { val lm = LanguageModel() val prefix = "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } ⭐ 72 tokens 67
  25. class LanguageModel { fun predict(prefix: String) { ... } }

    fun testGeneration() { val lm = LanguageModel() val prefix = "My dog " val expected = "is a golden retriever" val generation = lm.predict(prefix) assertEquals(generation, expected) } ⭐ 72 tokens 🦙 96 tokens 68