Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Breaking Tradeoffs: Extremely Scalable Multi-Ag...

Breaking Tradeoffs: Extremely Scalable Multi-Agent Pathfinding Algorithms

Used in a guest talk in the “Multi-Robot Planning and Coordination” course, Robotics Institute, Carnegie Mellon University (CMU)
https://jiaoyangli.me/teaching/2024-spring-16891

Based on the Cambridge talk (2023), but the CMU talk goes deeper.
https://speakerdeck.com/kei18/pathfinding-for-10k-agents-5534305f-45e3-4712-9605-ef112be6a7c5

Keisuke Okumura | 奥村圭祐

February 13, 2024
Tweet

More Decks by Keisuke Okumura | 奥村圭祐

Other Decks in Research

Transcript

  1. Breaking Tradeoffs: Extremely Scalable Multi-Agent Pathfinding Algorithms Keisuke Okumura1,2 7th

    Feb. 2024 “Multi-Robot Planning and Coordination” Robotics Institute, Carnegie Mellon University 1University of Cambridge 2National Institute of Advanced Industrial Science and Technology (AIST), Japan kei18 https://kei18.github.io [email protected] image generated by DALL-E 3
  2. /89 2 AP Photo/Ross D. Franklin Ross D. Franklin /

    AP Photo "Swarm" automation is ubiquitous
  3. /89 3 given agents graph goals solution paths without collisions

    cost total travel time, distance, makespan, etc MAPF: Multi-Agent Path Finding
  4. /89 4 Scope of This Talk solution quality optimal suboptimal

    optimal methods: A*, CBS, M*, BCP, SAT-based bounded suboptimal methods: (E)ECBS, I-ODrM* unbounded suboptimal methods: PP, LNS2 extremely scalable methods: PIBT, LaCAM* ≥1000 agents in seconds ≤100 agents in minutes speed & scalability tradeoff holy grail
  5. /89 7 vIdeo by Philip Cheung / New York Times

    Intel’s Automated Material-Handling System Reason-1: Demands for 1k-10k scale
  6. /89 9 Reason-3: Real world is more complicated rotation aggressive

    moves sometimes unavailable task interaction +heterogeneity what the industry expects typical research to keep MAPF alive: increase X or decrease ||X – Y|| e.g., X=10k! algorithms work with X agents with Y (≪ X) real agents
  7. /89 11 Hart, P. E., Nilsson, N. J., & Raphael,

    B. A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. on Systems Science and Cybernetics. 1968. start goal A* Search start goal f = g + h search tree
  8. /89 12 … … … … … search node (configuration)

    goal configuration Vanilla A* for MAPF
  9. /89 13 Complete algorithms return solutions for solvable instances in

    finite time; otherwise, they report the non-existence. Optimal algorithms always return solutions having minimum costs. Algorithm Properties A* is complete. It is optimal with admissible heuristics.
  10. /89 14 runtime (sec) solved instances (%)   

                - 13,900 instances - 33 grid maps - every 50 agents, up to max. (1000) - tested on standard desktop PC Stern, R. et al. Multi-Agent Pathfinding: Definitions, Variants, and Benchmarks. SoCS. 2019. 33 grid maps e.g., random-32-32-20, 200 agents Evaluation on MAPF Benchmark maze-32-32-2, 100 agents 00.0% A* [Hart+ 68] complete optimal algorithm properties computation time
  11. /89 15 Reason for poor performance of A* start goal

    search tree branching factor (number of successor nodes) O(5^N) N: #agents MAPF has a huge branching factor: 3,125 9,765,625 95,367,431,640,625 931,322,574,615,478,515,625 9,094,947,017,729,282,379,150,390,625 88,817,841,970,012,523,233,890,533,447,265,625 5^5 5^10 5^20 5^30 5^40 5^50 For 10k agents? Ridiculous!
  12. /89 16        

           runtime (sec) solved instances (%) 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] complete optimal algorithm properties A* variant
  13. /89 18 Relaxing Completness Optimal algorithms always return solutions having

    minimum costs. Complete algorithms return solutions for solvable instances in finite time; otherwise, they report the non-existence. unable to identify unsolvable instances
  14. /89 19 CBS: Conflict-based Search Sharon, G., Stern, R., Felner,

    A., & Sturtevant, N. R. Conflict-based search for optimal multi-agent pathfinding. AIJ. 2015. high-level search Image by GraphicMama-team from Pixabay identify conflicts in solution candidates low-level search find a path satisfying constraints (e.g., A*) query a single-agent path that avoids detected conflicts return a path satisfying constraints
  15. /89         

                         20 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] complete optimal runtime (sec) solved instances (%) 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] solution complete optimal (unable to identify unsolvable instances) two-level, but with mathematical optimization at high-level
  16. /89 21 Relaxing Optimality Optimal algorithms always return solutions having

    minimum costs. Complete algorithms return solutions for solvable instances in finite time; otherwise, they report the non-existence. allowing bounded suboptimal solutions: obtained solution cost ≤ w*(optimal solution cost) where w ≥ 1
  17. /89         

                         22 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] complete solution complete optimal optimal (unable to identify unsolvable instances) runtime (sec) solved instances (%) 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete bounded suboptimal A* variant
  18. /89 23 Relaxing Completeness and Optimality Optimal algorithms always return

    solutions having minimum costs. Complete algorithms return solutions for solvable instances in finite time; otherwise, they report the non-existence. allowing bounded suboptimal solutions
  19. /89 24 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15]

    8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete solution complete complete bounded suboptimal optimal optimal (unable to identify unsolvable instances) runtime (sec) solved instances (%)                50.5% EECBS-5 [Li+ AAAI-21] solution complete bounded suboptimal CBS variant
  20. /89 25 Give up Everything Optimal algorithms always return solutions

    having minimum costs. Complete algorithms return solutions for solvable instances in finite time; otherwise, they report the non-existence.
  21. /89 26 PP: Prioritized Planning Erdmann, M., & Lozano-Perez, T.

    On multiple moving objects. Algorithmica. 1987; Silver, D. Cooperative pathfinding. AIIDE. 2005. simple, quick, scalable, reasonable solution quality 1 stay 2 2. Perform single-agent pathfinding for each agent according to priorities, while avoiding collisions with already competed paths. 1 2 1. Assign priorities to each agent.
  22. /89 27 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15]

    8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete solution complete complete bounded suboptimal optimal optimal (unable to identify unsolvable instances) runtime (sec) solved instances (%)                50.5% EECBS-5 [Li+ AAAI-21] solution complete bounded suboptimal 61.4% PP [Silver AIIDE-05] incomplete suboptimal
  23. /89 28 MAPF-LNS2: Large Neighborhood Search Li, J., Chen, Z.,

    Harabor, D., Stuckey, P. J., & Koenig, S. MAPF-LNS2: fast repairing for multi-agent path finding via large neighborhood search. AAAI. 2022. high-level search low-level search query paths for the subset of agents return paths identify subset of agents (e.g., random selection) find paths for the subset with a smaller number of collisions within the subset (e.g., PP)
  24. /89 29 runtime (sec) solved instances (%)   

                0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete solution complete complete bounded suboptimal optimal optimal (unable to identify unsolvable instances) 50.5% EECBS-5 [Li+ AAAI-21] solution complete bounded suboptimal 61.4% PP [Silver AIIDE-05] incomplete suboptimal 80.9% LNS2 [Li+ AAAI-22] capable of addressing hundreds of agents
  25. /89 30 runtime (sec) solved instances (%)   

                0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete solution complete complete bounded suboptimal optimal optimal (unable to identify unsolvable instances) 50.5% EECBS-5 [Li+ AAAI-21] solution complete bounded suboptimal 61.4% PP [Silver AIIDE-05] 80.9% LNS2 [Li+ AAAI-22] lose nice props. Who cares theoretical properties? incomplete suboptimal
  26. /89 31 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15]

    8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] 50.5% EECBS-5 [Li+ AAAI-21] 61.4% PP [Silver AIIDE-05] 80.9% LNS2 [Li+ AAAI-22] complete solution complete complete solution complete bounded suboptimal bounded suboptimal optimal optimal (unable to identify unsolvable instances) incomplete suboptimal Scalable methods are "scalable"? 4 agents success?
  27. /89 32 @yr_6001_as / X Skylark Channel / YouTube Delivery

    Robots in Restaurants Deadlock Adversarial situations happen in reality
  28. /89 33 Summary so far (~2022) complete optimal incomplete suboptimal

    holy grail tradeoff theoretical properties planning effort small large speed & scalability
  29. /89 34 LaCAM* with PIBT (2023) complete optimal incomplete suboptimal

    planning effort small large speed & scalability theoretical properties generated by DALL-E 3
  30. /89 36 Unblock Me / Google Play DavidPlays / YouTube

    goal planning stage acting stage Two Styles to Solve Puzzle long-horizon (deliberative, offline) short-horizon (reactive, online)
  31. /89 37 theoretical guarantees planning effort small large speed &

    scalability complete optimal incomplete suboptimal long-horizon (deliberative, offline) short-horizon (reactive, online) Planning Horizon planning stage acting stage
  32. /89 38 theoretical guarantees planning effort small large speed &

    scalability Strategy to overcome the tradeoff short-horizon planning pulls long-horizon planning integration
  33. /89 39 This is just a metaphor. Image by OpenClipart-Vectors

    from Pixabay PIBT Okumura+ AIJ-22 LaCAM* Okumura AAAI-23, IJCAI-23
  34. /89 40 This is just a metaphor. Image by OpenClipart-Vectors

    from Pixabay PIBT Okumura+ AIJ-22 LaCAM* Okumura AAAI-23, IJCAI-23
  35. /89 41 PIBT: Priority Inheritance with Backtracking Okumura, K., Machida,

    M., Défago, X., & Tamura, Y. Priority Inheritance with Backtracking for Iterative Multi-agent Path Finding. AIJ. 2022. (extended from IJCAI-19*) collision-free configuration while reflecting preferences PIBT 1 desired 2 3 4 5 4 2 1 desired 3 *originally developed for lifelong pathfinding scenarios preference & priority + configuration High Low
  36. /89 44 high low mid as high priority inheritance Sha+

    IEEE Trans Comput-90 How PIBT works – 2/4
  37. /89 45 high as high as high as high as

    high stuck but still not feasible How PIBT works – 3/4
  38. /89 46 invalid valid re-plan re-plan valid You can move

    invalid You must re-plan, I will stay introduce backtracking How PIBT works – 4/4 always generate collision-free configurations
  39. /89 47 Performance of PIBT random-32-32-20 32x32 30sec timeout 

              #agents success rate EECBS PP LNS2 0% PIBT runtime (sec)          #agents EECBS PP ost003d 194x194 four-connected grid LNS2 blazing fast! worst: 550ms PIBT quick but shortsighted
  40. /89         

          48 runtime (sec) solved instances (%)                0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] complete solution complete complete bounded suboptimal optimal optimal (unable to identify unsolvable instances) 50.5% EECBS-5 [Li+ AAAI-21] solution complete bounded suboptimal 61.4% PP [Silver AIIDE-05] incomplete suboptimal 80.9% LNS2 [Li+ AAAI-22] 67.4% PIBT [Okumura+ AIJ-22] PIBT is convenient, blazing fast, but…
  41. /89 49 This is just a metaphor. Image by OpenClipart-Vectors

    from Pixabay PIBT Okumura+ AIJ-22 LaCAM* Okumura AAAI-23, IJCAI-23
  42. /89 50 … … … … … search node (configuration)

    goal configuration Recap: A* exponential number of node generation greedy search: 44 nodes
  43. /89 51 PIBT initial configuration Recap: PIBT use PIBT to

    guide exhaustive search only 4 configurations PIBT goal configuration PIBT
  44. /89 52 … … PIBT initial configuration … … PIBT

    goal configuration Okumura, K. LaCAM: Search-Based Algorithm for Quick Multi-Agent Pathfinding. AAAI. 2023 LaCAM: Lazy Constraints Addition Search for MAPF … PIBT not generated immediately 1. Configurations are generated in a lazy manner exhaustive search with two tricks 2. Use other MAPF algorihtms to generate a promising configuration greedy: 44 nodes LaCAM: 4 nodes => quick & complete MAPF
  45. /89 must go left in the next config. 53 constraint

    tree (maintained implicitly) invoked multiple times during the search Lazy constraints addition – 1/4
  46. /89 56 e.g., breadth-first search 24th invoke configuration generation with

    Lazy constraints addition – 4/4 completeness proof: Each configuration eventually generates all neighbor configurations.
  47. /89 57        

                  EECBS PP LNS2 PIBT worst: 11sec LaCAM #agents success rate random-32-32-20, 32x32, 30sec timeout, 400 agents Performance of LaCAM
  48. /89         

          58                runtime (sec) solved instances (%) 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15] 8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] 50.5% EECBS-5 [Li+ AAAI-21] 61.4% PP [Silver AIIDE-05] 80.9% LNS2 [Li+ AAAI-22] 67.4% PIBT [Okumura+ AIJ-22] complete solution complete complete solution complete bounded suboptimal bounded suboptimal optimal optimal (unable to identify unsolvable instances) incomplete suboptimal 85.6% LaCAM [Okumura AAAI-23] complete suboptimal Start breaking the tradeoff!
  49. /89 59 0.0% A* [Hart+ 68] 0.4% ODrM* [Wagner+ AIJ-15]

    8.3% CBS [Sharon+ AIJ-15; Li+ AIJ-21] 10.7% BCP [Lam+ COR-22] 30.9% I-ODrM*-5 [Wagner+ AIJ-15] 50.5% EECBS-5 [Li+ AAAI-21] 61.4% PP [Silver AIIDE-05] 80.9% LNS2 [Li+ AAAI-22] complete solution complete complete solution complete bounded suboptimal bounded suboptimal optimal optimal (unable to identify unsolvable instances) 4 agents 85.6% LaCAM [Okumura AAAI-23] complete suboptimal incomplete suboptimal 67.4% PIBT [Okumura+ AIJ-22] Start breaking the tradeoff!
  50. /89 60        

           runtime (sec) solved instances (%) 85.6% LaCAM [Okumura AAAI-23] complete suboptimal Missing pieces generated by DALL-E 3
  51. /89 61 configuration & cost (e.g., makespan) 1 2 3

    4 6 5 0 initial config. 5 goal config. LaCAM stops the search when finding the goal config. search tree parent – children other neighbors Recap: LaCAM
  52. /89 62 1 2 3 4 5 6 0 5

    LaCAM* continues the search after finding the goal config. parent – children other neighbors goal config. search tree 1 initial config. configuration & cost (e.g., makespan) LaCAM* Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.
  53. /89 63 1 2 3 4 5 6 0 5

    1 new edge when finding new connections, LaCAM* Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.
  54. /89 64 1 2 3 3 2 3 0 4

    1 when finding new connections, rewrite the tree by Dijkstra This is an anytime algorithm that eventually converges to the optimal solution if the solution cost is accumulative transition costs LaCAM* Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.
  55. /89 65        

           runtime (sec) solved instances (%) 85.6% LaCAM* complete eventually optimal fine-tuning PIBT
  56. /89         

          66                runtime (sec) solved instances (%) 85.6% LaCAM + PIBT complete suboptimal 99.0% LaCAM*+ fine-tuned PIBT complete eventually optimal Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.
  57. /89 67        

           runtime (sec) solved instances (%) 99.0% LaCAM* (initial solution) complete eventually optimal LaCAM* establishes a landmark result! Okay, it’s too crazy... remaining 1%: only maze-128-128-1 Okumura, K. Improving LaCAM for Scalable Eventually Optimal Multi-Agent Pathfinding. IJCAI. 2023.
  58. /89 69 Can we build scalable MAPF algorithms, while still

    having nice theoretical guarantees?
  59. /89 72 Is LaCAM* the universally dominant MAPF solver? No.

    solution quality ≥1000 agents in seconds ≤100 agents in minutes optimal suboptimal optimal methods: A*, CBS, M*, BCP, SAT-based bounded suboptimal methods: (E)ECBS, I-ODrM* unbounded suboptimal methods: PP, LNS2 extremely scalable methods: PIBT, LaCAM* speed & scalability eventually optimal; hopeless with large instances
  60. /89 73    flowtime+ / LB instance x25

    LNS2 random-32-32-20 409 agents, 30s timeout No. vanilla LaCAM* – initial solution +LaCAM* aims to optimize sum-of-loss (≠ flowtime); check the paper for details. Is LaCAM* the universally dominant MAPF solver?
  61. /89 74    instance x25 LNS2 random-32-32-20 409

    agents, 30s timeout No. vanilla LaCAM* – initial solution vanilla LaCAM* – after 30s +LaCAM* aims to optimize sum-of-loss (≠ flowtime); check the paper for details. flowtime+ / LB Is LaCAM* the universally dominant MAPF solver?
  62. /89 75 solution quality optimal suboptimal ≥1000 agents in seconds

    ≤100 agents in minutes speed & scalability holy grail Towards Real-Time, Large-Scale, and Near-Optimal MAPF Vanilla LaCAM* is a good starting point
  63. /89 76 initial solution planning deadline 1. improve initial solution

    quality 2. improve convergence speed Improving Anytime Planning vanilla LaCAM* computation time solution cost solution
  64. /89 78 Engineered LaCAM* LaCAM* fine-tuned PIBT Monte-Carlo sampling dynamic

    incoporation of alternative solutions recursive LaCAM* non-deterministic node extraction with space utilization optimization improve initial solution quality improve convergence speed particularly effective Okumura, K. Engineering LaCAM∗: Towards Real-Time, Large-Scale, and Near-Optimal Multi-Agent Pathfinding. AAMAS. 2024.
  65. /89 79 SUO: Space Utilization Optimization original idea from: Han,

    S.,& Yu, J. Optimizing space utilization for more effective multi-robot path planning. ICRA. 2022. Also check: Chen, Z., Harabor, D., Li, J., & Stuckey, P. J. Traffic Flow Optimisation for Lifelong Multi-Agent Path Finding. AAAI. 2024. configuration, priority, and preference collision-free configuration PIBT Recap: 1 (desired) 2 3 4 5 Let’s optimize the preference b a High Low cost:3+2 cost:3+3 b a Vanilla PIBT either chooses ‘a’ or ‘b’ blindly => significant effect with more agents
  66. /89 80 SUO: Space Utilization Optimization original idea from: Han,

    S.,& Yu, J. Optimizing space utilization for more effective multi-robot path planning. ICRA. 2022. Also check: Chen, Z., Harabor, D., Li, J., & Stuckey, P. J. Traffic Flow Optimisation for Lifelong Multi-Agent Path Finding. AAAI. 2024. Step 1: Compute spatially dispersed paths Collisions are allowed Pickup an agent and recompute its path while minimizing collisions      runtime (sec) cost / LB vanilla LaCAM* LaCAM* with SUO 12% random-32-32-20, 409 agents Step 2: Run PIBT, while making agents follow the precomputed paths as much as possible
  67. /89 81 computation time solution cost planning deadline vanilla LaCAM*

    Dynamic Incorporation of Alternative Solutions Surynek, P. Redundancy elimination in highly parallel solutions of motion coordination problems. IJAIT. 2013. De Wilde, B., Ter Mors, A. W., & Witteveen, C. Push and rotate: a complete multi-agent pathfinding algorithm. JAIR. 2014. Okumura, K., Tamura, Y., & Défago, X. Iterative refinement for real-time multi-robot path planning. IROS. 2021. Li, J., Chen, Z., Harabor, D., Stuckey, P. J., & Koenig, S. Anytime multi-agent path finding via large neighborhood search. IJCAI. 2021. … Effective local repair methods exist to improve solution quality: local repair (parallel run) incorporate solutions and continue the search
  68. /89 82 when alternative solutions found, configuration & cost (e.g.,

    makespan) parent – children other neighbors Dynamic Incorporation of Alternative Solutions 1 2 3 3 2 3 0 4 1 initial config. search tree goal config.
  69. /89 83 2 Dynamic Incorporation of Alternative Solutions 1 2

    3 3 2 3 0 3 1 when alternative solutions found, incorporate them with tree rewriting      runtime (sec) cost / LB vanilla LaCAM* with the dynamic incorporation (LNS-based refinement) 13% random-32-32-20, 409 agents This scheme is more powerful than it looks: It is theoretically possible to eventually find optimal solutions from arbitrary suboptimal solutions
  70. /89 85    instance x25 vanilla LaCAM* –

    initial solution LNS2 random-32-32-20 409 agents, 30s timeout vanilla LaCAM* – after 30s engineered LaCAM* – after 30s engineered LaCAM* – initial solution flowtime+ / LB ~30% improvement! Performance of Engineered LaCAM* Can we do more? I believe so… +LaCAM* aims to optimize sum-of-loss (≠ flowtime); check the paper for details.
  71. /89 86 Takeaways Powerful MAPF methods are by combinations of

    several MAPF methods. The frontiers are gradually approaching real-time, large-scale, and near-optimal MAPF, but are still far away. Many research opportunities exist! Good theoretical properties and scalability can coexist!
  72. /89 87 Jan. 2021. When I was a 1st-year PhD

    student: Pathfinding for 10k agents? You’re a dreamer! To be honest, I agreed at the time.
  73. /89 88 Jan. 2021. When I was a 1st-year PhD

    student: Pathfinding for 10k agents? You’re a dreamer! To be honest, I agreed at the time. LaCAM* 2023 Image by naobim from Pixabay
  74. /89 90 Thank you for listening! Acknowledgements to mentors /

    collaborators: X. Defago, Y. Tamura, F. Bonnet, M. Machida, R. Yonetani, M. Nishimura, A. Kanezaki, S. Tixeuil A. Prorok + members of PROROK Lab in Cambridge Funding: JSPS DC & Overseas Research Fellowship, JST ACT-X, Yoshida Scholarship Foundation And, J. Li for this exciting opportunity & her inspiring studies! kei18 https://kei18.github.io [email protected] Questions / research collaboration proposals are welcome: