from movement sequences pose challenges in various domains 2 Other: pedestrians, vehicles ….. Discovering the directed interaction will contribute to the understanding of the principles of biological agents' behaviors Humans (in basketball) Animals (bats)
a practical framework for exploratory analysis in various fields • Recently: inferring GC under nonlinear dynamics [Tank+18; Khanna+19] Problem: the structure of the generative process in biological multi- agent trajectories, which include time-varying dynamical systems, is not fully utilized in existing base models of GC (e.g., VAR and NN) 1. Ignoring the structures of such processes will lead to interpretational problems and sometimes erroneous assessments of causality solution: incorporating the structures into the base model for inferring GC, e.g., augmenting (inherently) incomplete behavioral models with interpretable data-driven models, can solve these problems 2. Data-driven models sometimes detect false causality that is counterintuitive to the user of the analysis solution: introducing architectures and regularization to utilize scientific knowledge will be effective for a reliable base model of a GC method 3
of ABM with NN inferring time-varying GC 1. Formulation of augmented behavioral model (ABM) (sec. 3.2) 2. Learning of ABM (sec. 4.1) 4. Inference of Granger causality (sec. 4.3) 3. Theory-guided regularization (sec. 4.2) (sec. is the reference to our paper)
Conceptual animal behavioral model (not numerically computable) Sign of GC (navigation) e.g., attraction and repulsion positive weights of GC (motion) Augmented Behavioral model (computable and interpretable) concatenated It is closely related to self-explanatory NN [Alvarez-Melis & Jaakkola, 18] (sec. 3.3)
regularization term (iv) smoothing penalty term Theory-guided weight (given in the next slide) Concatenated weight matrix Loss function (ii) sparsity-inducing penalty term Learn using MLP where
known scientific knowledge [Karpatne+ 17] (mainly studied on physical principles) • Our basic idea: we utilize theory-based and data-driven prediction results and impose penalties in the appropriate situations 1. let ෝ 𝒙𝑡 be the prediction from the data 2. prepare some input-output pairs ( 𝒙𝑡−𝑘≤𝑡 , 𝒙𝑡 ) based on scientific knowledge • assume that the weight 𝚿𝑡 𝑇𝐺 is uniquely determined • this assumption reduces the possible pairs ( 𝒙𝑡−𝑘≤𝑡 , 𝒙𝑡 ) 3. When ෝ 𝒙𝑡 and 𝒙𝑡 are similar, impose penalties on the weights such that the cause of ෝ 𝒙𝑡 (i.e., 𝚿𝑡 ) is similar to the cause of 𝒙𝑡 (i.e., 𝚿𝑡 𝑇𝐺). • we assume that the cause of ෝ 𝒙𝑡 is equivalent to the cause of 𝒙𝑡 at the time Here we utilize the only intuitive prior knowledge such that the agents go straight from the current state if there are no interactions 7
A variable 𝑥𝑖 does not Granger-cause variable 𝑥𝑗, denoted as 𝑥𝑖 ↛ 𝑥𝑗, if and only if the prediction model of 𝑥𝑗 is constant in 𝑥≤𝑡 𝑖 . 8 (Wikipedia) 𝑥𝑖 𝑥𝑗 We here consider GC using the obtained In the following equation: We consider 𝑆𝑖,𝑗,𝑡 ≈ 0 to be non-causal relationships and 𝑆𝑖,𝑗,𝑡 ≫ 0 if 𝑥𝑖 → 𝑥𝑗 𝑑: output dim. 𝑑𝑟 : input dim. for each agent (e.g., 2D or 3D) signmax: sign of the larger value of max and min (e.g., signmax({1, 2, −3}) = −1) is the Frobenius norm of the matrix
rules: attraction, repulsion, and alignment Experiments (2) Boid model (synthetic data) [Khanna+19] [Löwe+20] [Marcinkevics+20] w/o regularization w/o learning of sign Here we set the boids directed preferences: true relations 1, 0, and −1 as attraction, no interaction, and repulsion Experimental results: both learning of sign and TG regularization were needed e.g., #1 attracts #5 (+1) and is avoid by #3 (-1)
positive: attraction negative: repulsion Experiments (3) real-world mice data Raised in the same cage • Our method extracted a larger duration in the different cage than that in the same, whereas GVAR did too much interaction. • Our methods characterized the movement behaviors before the contacts with others [Thanos+17] (birds, bats, and flies are in our paper)
via ABM, which can extract interaction rules from real-world multi-agent and multi-dimensional trajectory data • We realized the theory-guided regularization for reliable biological behavioral modeling, which can leverage scientific knowledge such that “when this situation occurs, it would be like this” • Biologically, we reformulate a well-known conceptual behavioral model, which did not have a numerically computable form, such that we can compute and quantitatively evaluate it • Our method achieved better performance than various baselines using synthetic datasets, and obtained new biological insights using multiple datasets of mice, birds, bats, and flies 13 Acknowledgments: This work was supported by JSPS KAKENHI (Grant Numbers 19H04941, 20H04075, 16H06541, 25281056, 21H05296, 18H03786, 21H05295, 19H04939, JP18H03287, and 21H05300), JST PRESTO (JPMJPR20CA), and JST CREST (JPMJCR1913). For obtaining flies data, we would like to thank Ryota Nishimura at Nagoya Univ.