Physics-Informed Machine Learning For Sound Field Estimation and Control

Physics-Informed Machine Learning For Sound Field Estimation and Control Shoichi
Koyama National Institute of Informatics, Tokyo, Japan

About me Ø Shoichi Koyama, Ph.D. • 2009 ‒ 2014:
Researcher, NTT Labs • 2014 ‒ 2018: Research Associate, UTokyo • 2016 ‒ 2018: Visiting Researcher, Paris Diderot Univ. • 2018 ‒ 2023: Lecturer, UTokyo • 2020 ‒ 2023: Visiting Associate Prof., Tohoku Univ. • 2023 ‒ current: Associate Prof., NII September 3, 2024 2

About NII Ø NII is national research institute of informatics
in Japan – Main lab is located in central Tokyo – Associated with graduate university called SOKENDAI September 3, 2024 3 Kashiwa Annex NII 29 km Join us as an intern or PhD student!

September 3, 2024 4 Basic Technologies of Sound Field Estimation
and Control VR/AR audio Active noise control Local-field recording and reproduction Signal enhancement Visualization/auralization Room acoustic analysis Sound field estimation/control and its applications

What is sound field estimation/control? September 3, 2024 5 Estimating
sound field inside target region using multiple mics Synthesizing desired sound ﬁeld inside target region using multiple loudspeakers Physics-informed signal processing/machine learning for sound field estimation and control Estimation Control Microphone Loudspeaker

SOUND FIELD ESTIMATION September 3, 2024 6

Sound field estimation September 3, 2024 7 Formulation of sound
field estimation problem Estimate pressure distribution with observations at discrete set of mics in the frequency domain : Source-free and simply-connected interior region Microphone Target region:

Sound field estimation Ø General interpolation techniques – Representing by
model parameters as – Solve the optimization problem September 3, 2024 8 Formulation of sound field estimation problem Microphone Target region: <latexit sha1_base64="QB9up2Xq4KtGilt/H+nRpzcDgKk=">AAADJnicfVLdahQxFM6MP63rT3f10pvgImx1WWZEqiBCUS+8sFjFbQubdchkz8yGJjNDkhHXkKfwJXwa70S80zcxM93qtrUeCHx85zv/SSvBtYmiH0F44eKly2vrVzpXr12/sdHt3dzTZa0YjFkpSnWQUg2CFzA23Ag4qBRQmQrYTw+fN/79D6A0L4t3ZlHBVNK84Bln1Hgq6VoiecEl/wSJJam0xMzBUOcwkdTMGRX2lSMCMjPAxOJ60GiUS+QTvKLexMQlVj6N3fudYevQPoHi+dxs4vt/U711g9WopNuPRlFr+CyIl6CPlrab9ILPZFayWkJhmKBaT+KoMlNLleFMgOuQWkNF2SHNYeJhQSXoqW235PBdz8xwVir/CoNbdjXCUqn1QqZe2TSsT/sa8l++SW2yx1PLi6o2ULCjQlktsClxs3I84wqYEQsPKFPc94rZnCrKjD9M50SZVA6b/Epn2k/zAvyUCnY887oCRU2p7llCVS7pR+enzsmwQf8T8uJY6NF5Qp+k/QLO/kHnSpe/xUuPkT9jfPpoZ8Heg1G8Ndp687C//Wx50HV0G91BAxSjR2gbvUS7aIwY+hWsBd2gF34Jv4bfwu9H0jBYxtxCJyz8+RuXLAcP</latexit> minimize ✓ L {u(rm; ✓)}M m=1 , s + R(✓) <latexit sha1_base64="bgfM3WFHH+06JgGQdjycQWMo7ww=">AAACwXicfZHBbhMxEIadhUIILU1A6oWLRVSprapoF6GCxKWiPfSCaKWmrZSNollnNrFqe1f2LCIs+xS8AVd4Id6m3jRBtKWMZPnTP//YHk+SK+koDH83ggcPVx49bj5pPV1de7be7jw/c1lhBfZFpjJ7kYBDJQ32SZLCi9wi6ETheXJ5UOfPP6N1MjOnNMtxqGFiZCoFkJdG7Y1iK050aav3vN5jmiJBtT1qd8NeOA9+F6IFdNkijkedxvd4nIlCoyGhwLlBFOY0LMGSFAqrVlw4zEFcwgQHHg1odMNy3kDFN70y5mlm/TLE5+rfFSVo52Y68U4NNHW3c7X4r9ygoPTdsJQmLwiNuL4oLRSnjNe/wcfSoiA18wDCSv9WLqZgQZD/s9aNaxK9W59vXep8N4fou7T40SufcrRAmd0pY7ATDV8q3/Uk3q3pf0ZplkZP9xn9IVLLr1iVf+heqzRL65L8GKPbQ7sLZ6970V5v7+RNd//DYqBN9pK9YlssYm/ZPjtix6zPBPvGfrCf7FdwEMggD+y1NWgsal6wGxGUV4wM4Gk=</latexit> u(r; ✓) <latexit sha1_base64="yLksvcVXn4jdtyNYM7Soz6yW9Ko=">AAACqnicfZHbSgMxEIbT9VxPrV56s1gEkVJ2RaqXRb3wRjxgq9IWmU1nazDJLklWrEufwEv14Xwbs7UVrYeBwMc//yQzmSDmTBvPe8s5E5NT0zOzc/n5hcWl5UJxpaGjRFGs04hH6ioAjZxJrBtmOF7FCkEEHC+Du4Msf3mPSrNIXphejG0BXclCRsFY6Sy5KZS8ijcI9yf4QyiRYZzeFHPPrU5EE4HSUA5aN30vNu0UlGGUYz/fSjTGQO+gi02LEgTqdjrotO9uWKXjhpGyRxp3oH6tSEFo3ROBdQowt3o8l4m/5ZqJCffaKZNxYlDSj4fChLsmcrOx3Q5TSA3vWQCqmO3VpbeggBr7OflvzwSinN2vdKjtNIdop1R4bJWTGBWYSG2lLVBdAQ99O3W3Vc7oPyOTI6Olv4z2EibYI/bTT/rTyuTIOiK7Rn98aT+hsV3xq5Xq2U6ptj9c6CxZI+tkk/hkl9TIETkldUIJkifyQl6dsnPuXDvND6uTG9askm/hdN4Bu6fXkA==</latexit> u <latexit sha1_base64="ru0OGkVwXcC9n3iBv3AvtKvDAnk=">AAACtHicfZFdSxtBFIYnW1vTaFttL71ZDAURCbtSopeh7YU3RQtGhWyQs5OzyeB8LDNnxbjsryi9rb+r/6azMRE/e2Dg4T3vzJyPNJfCURT9bQSvll6/WW6+ba2svnv/YW3944kzheXY50Yae5aCQyk09kmQxLPcIqhU4ml68a3On16idcLoY5rmOFQw1iITHMhLgyRVZUITJKjO19pRJ5pF+BTiObTZPI7O1xu/kpHhhUJNXIJzgzjKaViCJcElVq2kcJgDv4AxDjxqUOiG5azmKvzslVGYGeuPpnCm3r9RgnJuqlLvVEAT9zhXi8/lBgVl+8NS6Lwg1Pz2o6yQIZmwHkA4EhY5yakH4Fb4WkM+AQuc/JhaD75J1U79vnWZ8918R9+lxR9eOczRAhm7XSZgxwquKt/1ONmp6X9GoRdGTy8Z/SNCiWusyjt60Sr0wrogv8b48dKewsluJ+52uj+/tHtf5wttsg22ybZYzPZYjx2wI9ZnnBn2m/1hN0E3SAIe4K01aMzvfGIPItD/AKw13BQ=</latexit> ✓ Loss function for observation Regularization term for <latexit sha1_base64="ru0OGkVwXcC9n3iBv3AvtKvDAnk=">AAACtHicfZFdSxtBFIYnW1vTaFttL71ZDAURCbtSopeh7YU3RQtGhWyQs5OzyeB8LDNnxbjsryi9rb+r/6azMRE/e2Dg4T3vzJyPNJfCURT9bQSvll6/WW6+ba2svnv/YW3944kzheXY50Yae5aCQyk09kmQxLPcIqhU4ml68a3On16idcLoY5rmOFQw1iITHMhLgyRVZUITJKjO19pRJ5pF+BTiObTZPI7O1xu/kpHhhUJNXIJzgzjKaViCJcElVq2kcJgDv4AxDjxqUOiG5azmKvzslVGYGeuPpnCm3r9RgnJuqlLvVEAT9zhXi8/lBgVl+8NS6Lwg1Pz2o6yQIZmwHkA4EhY5yakH4Fb4WkM+AQuc/JhaD75J1U79vnWZ8918R9+lxR9eOczRAhm7XSZgxwquKt/1ONmp6X9GoRdGTy8Z/SNCiWusyjt60Sr0wrogv8b48dKewsluJ+52uj+/tHtf5wttsg22ybZYzPZYjx2wI9ZnnBn2m/1hN0E3SAIe4K01aMzvfGIPItD/AKw13BQ=</latexit> ✓ <latexit sha1_base64="wnDK9ytRrHjhZwyUfqPJAj2Ykik=">AAAC03icfZHLahRBFIZrOlHjeJvo0k3hIAQZhm6R6EYIxoWbYIRMEphum9M1pydF6tJUVUsmRW/ErW58Fbf6Hr6N1ZOZYBLjgYKP//xVdS5FJbh1cfy7E62s3rh5a+12987de/cf9NYf7ltdG4YjpoU2hwVYFFzhyHEn8LAyCLIQeFAcb7f5g09oLNdqz80qzCRMFS85AxekvLeRFtLbhr6mY5snA5qKiXZ2QG2+k330qQR3ZEu/1zR5rx8P43nQq5AsoE8WsZuvd76nE81qicoxAdaOk7hymQfjOBPYdNPaYgXsGKY4DqhAos38vKWGPg3KhJbahKMcnat/3/AgrZ3JIjjnNV7OteK/cuPala8yz1VVO1Ts7KOyFtRp2s6HTrhB5sQsADDDQ62UHYEB5sIUuxe+KeSgfd/Y0oZu3mLo0uBOUN5XaMBp88ynYKYSTprQ9TQdtPQ/I1dLY6DrjOERLvkpNv6crrVytbQuKawxuby0q7D/fJhsDjc/vOhvvVksdI08Jk/IBknIS7JF3pFdMiKMfCM/yE/yKxpFPvocfTmzRp3FnUfkQkRf/wAenuda</latexit> s = [s1, . . . , sM ]T

Sound ﬁeld estimation Ø Function to be interpolated should satisfy
Helmholtz eq – Homogeneous Helmholtz equation in source-free target region – Conventional approach: expansion into element solutions of Helmholtz eq • Plane wave expantion (or Herglotz wave function) • Spherical wave function expansion • Equivalent source distribution (or single layer potential) September 3, 2024 9 What kind of physical properties can be embedded? <latexit sha1_base64="3MQnLgSPT9UXBsXZ4NqNl8viJ5k=">AAACynicfZHPThsxEMadhbY0/Rfg2EOtRpVoi6LdCAEXJEQ59FBUKjWAlA3RrDMbrNjele1FBGtv9Al4DK70Yfo29YakKlA6kqWfvvnG9swkueDGhuGvWjA3/+jxk4Wn9WfPX7x81VhcOjBZoRl2WCYyfZSAQcEVdiy3Ao9yjSATgYfJ6FOVPzxFbXimvttxjj0JQ8VTzsB6qd94sxIrSAT0XZxIp8vyuE0/0tFx+z0t6BYN+41m2AonQe9DNIUmmcZ+f7F2GQ8yVkhUlgkwphuFue050JYzgWU9LgzmwEYwxK5HBRJNz006Kek7rwxomml/lKUT9e8KB9KYsUy8U4I9MXdzlfivXLew6WbPcZUXFhW7eSgtBLUZrcZCB1wjs2LsAZjm/q+UnYAGZv3w6reeSeRqdb82qfHd7KLvUuOeV77mqMFm+oOLQQ8lnJW+62G8WtH/jFzNjJ4eMvpLuOTnWLo/9KCVq5l1Rn6N0d2l3YeDditab61/W2tu70wXukBek7dkhURkg2yTz2SfdAgjP8gVuSY/gy+BDsaBu7EGtWnNMrkVwcVvo0niRw==</latexit> (r2 r + k2)u = 0 Linear combination of element solutions is still a solution of Helmholtz eq

Basis expansion for sound ﬁeld estimation Ø Plane wave expansion
(or Herglotz wave function) September 3, 2024 10 <latexit sha1_base64="X5z1xMoVBReL3dH3nbvjSRQTg+g=">AAADMXicfZHNbhMxEMe9y1cJX2k5loNFhFRQiLIVKlyQKuDABVEEaSvFIfI6s1sT27uyvYhg+coL8Cg8TW+IKzwE3s0HtKWMZOnvmd94PDNpKbix/f5xFF+4eOnylbWrrWvXb9y81V7f2DdFpRkMWCEKfZhSA4IrGFhuBRyWGqhMBRyk0+d1/OAjaMML9c7OShhJmiuecUZtcI3bX6otkkqn/X38FBOu7NgRSe1Rmrq3frztMbFcTMBVvuEIWBrQBtHSgX/vHi4vHzyeYiKoygXgFdzF8/cx0U3Er5In/g81bnf6vX5j+KxIFqKDFrY3Xo++kknBKgnKMkGNGSb90o4c1ZazUKVFKgMlZVOawzBIRSWYkWsG5vG94JngrNDhKIsb798ZjkpjZjINZP1ZczpWO/8VG1Y2ezJyXJWVBcXmhbJKYFvgevp4wjUwK2ZBUKZ5+CtmR1RTZsOOWifKpLLbDMpkJnTzAkKXGl4Fz+sSNLWFfuAI1bmkn3zoOifdWv0P5GoJBnUeGB7hkn8G71bqXJSrJbpUYY3J6aWdFfvbvWSnt/PmUWf32WKha2gT3UVbKEGP0S56ifbQADH0K9qINqM78bf4OP4e/5ijcbTIuY1OWPzzN9y1DKA=</latexit> u(r) = Z S 2 ˜ u(⌘)e jkh⌘,rid⌘ -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 Plane wave arrival direction

Basis expansion for sound field estimation Ø Spherical wave function
expansion September 3, 2024 11 <latexit sha1_base64="GDd9eLOC2AN8O5PiWLeacMEt4Ac=">AAADinicfVFdb9MwFHUWPkYZ0MEjLxYVUou6KZ3QAKFKEwyJF8SQ6DZUl8pxndQ0diJ/IIqXX8Gv44m/wk3aIvbRWUp0dM655/r6xkUmjI2i38FGeOPmrdubdxp3t+7df9DcfnhscqcZH7A8y/VpTA3PhOIDK2zGTwvNqYwzfhLP3lb6yXeujcjVZzsv+EjSVIlEMGqBGjf/uDaJpddlB/cxMU6OPVGuH5VfPREqsfNyxUrX3wGpEuCPiaR2qoVKvSuXEZWpIqXPS8j7VkeV7Rk5W+g7V9jIWQd/qY1d6ACxGU9sG5NEU+bXl5X++lAI0iKd2s642Yp2o/rgy6C3BC20PEfj7eAXmeTMSa4sy6gxw15U2JGn2gqW8bJBnOEFZTOa8iFARSU3I19vosRPgZngJNfwKYtr9v8KT6UxcxmDs7quuahV5FXa0Nnk5cgLVTjLFVs0SlyGbY6rteKJ0JzZbA6AMi3grphNKTyiheU3zrWJZbd+KpMYmOaQw5SafwDmY8E1tbl+5gnVqaQ/Spg6Jd0KXWcUamUEtM4IIUKKn7z0/9Baq1Ar6wrBGnsXl3YZHO/t9vZ39z89bx28WS50Ez1GT1Ab9dALdIDeoyM0QCx4F8wCG7hwK9wLX4WvF9aNYFnzCJ074eFfxqEvpQ==</latexit> u(r) = 1 X ⌫=0 ⌫ X µ= ⌫ ˚ u(ro)j⌫(kkr ro k)Y⌫,µ ✓ r ro kr ro k ◆ -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 Expansion center Spherical Bessel function Spherical harmonic function

Basis expansion for sound field estimation Ø Linear regression with
finite-dimensional basis expansion – Truncation of spherical wave function expansion – Estimation of expansion coefs by linear ridge regression ( ) September 3, 2024 12 <latexit sha1_base64="0OkBaoq/FquuK30YCcYUCWtno4I=">AAADsXicfVJdb9MwFHUWBqN8bfDIi0UF6thWNQUNXpAmxsN4AIpEt6G5LY7rpFYdJ7IdRDH5E/AIf4x/w03ajq2lWIp0fO65956b6zCTwthW67e35l9Zv3pt43rtxs1bt+9sbt09NmmuGe+yVKb6NKSGS6F41wor+WmmOU1CyU/C8WEZP/nMtRGp+mAnGe8lNFYiEoxaoAZb3joZUetIQu1ICxU7EiYuL4oCP3qBCdVxItRgOUyEqrgwdIdF3zXe7gTb/TZkkW+4lJgC71WAdEYC2MV80PXbeAcTCVaHFK7LLUoFIbXSh+SRbfyt158aMpE7At15lwvlgHsNTbSIR3a77/aCYkX2zO1gs95qtqqDl0EwA3U0Ox34cT/IMGV5wpVlkhpzFrQy23NUW8EkL2okNzyjbExjfgZQ0YSbnqsWVuCHwAxxlGr4lMUVezHD0cSYSRKCsrK6GCvJf8XOchs97zmhstxyxaaNolxim+Jy+3goNGdWTgBQpgV4xWxENWUW3kjtUpsw2a1WYiID07ziMKXmb4B5l3FNbaofu+p90C8FTB2T3RL9TyjUXAholRCKiER85YU7RyulQs2lcwRrDBaXtgyO281gv7n//mn94OVsoRvoPnqAGihAz9ABOkId1EXMk95376f3y3/if/Q/+eFUuubNcu6hS8cf/wE+zjmX</latexit> ˆ ˚ u = arg min ˚ u2C(N+1)2 ks ˚ uk2 + k˚ uk2 = H + I 1 Hs <latexit sha1_base64="UTeURbIWpT12en/b/r7Bpp0fQFY=">AAADyHicfVLbbhMxEPV2uZRwa+ERHiwiqgS10QahgpAiVcADQgKK1LRFcRp5He/GZO1d+VIaXL/AJ/AVvMLX8Dd4NwnqLR1pV0dnzhx7ZhwXGVM6iv4GS+GVq9euL9+o3bx1+87dldV7uyo3ktAuybNc7sdY0YwJ2tVMZ3S/kBTzOKN78fh1md87pFKxXOzoSUH7HKeCJYxg7anBavDQNFDMrXRNuIZwUcj8CCJl+MAiYTqRO7Af3JzgprPhWc+Vf4g41iPJRGqNm5mUopLkNnfe8Uvl4hpjdDzNb1wgQ8dN+LkSrvsTvG1GE92AKJGY2MVlzl5u6o0kS0e6CRGqrb3swFKFDrEsRmx+3ebBtEYldsedbKhMG+cGK/WoFVUBz4P2DNTBLLb9PH+iYU4Mp0KTDCvVa0eF7lssNSMZdTVkFC0wGeOU9jwUmFPVt9UeHXzsmSFMcuk/oWHFnqywmCs14bFXVrc+myvJi3I9o5MXfctEYTQVZHpQYjKoc1g+CjhkkhKdTTzARDJ/V0hG2M9f+6dTO3VMzNerKalE+W7eUN+lpO8987GgEutcPrEIy5TjI+e7TtF6iS4TMjEXerRI6E0YZ9+os//RQikTc+kc+TW2zy7tPNh92mpvtjY/PatvvZotdBk8AI9AA7TBc7AF3oJt0AUk+BH8Cn4Hf8J3YRF+DSdT6VIwq7kPTkX4/R8XAkgu</latexit> u(r) ⇡ N X ⌫=0 ⌫ X µ= ⌫ ˚ u(ro)j⌫(kkr ro k)Y⌫,µ ✓ r ro kr ro k ◆ := '(r)T˚ u Basis function vector Expansion coef vector <latexit sha1_base64="FiFJVDnI/rcRBQL0/t8bc+E+oyY=">AAAC+3icfZHLbhMxFIad4VbCLS1LNhYRUouiaKZChQ1SBSzYVASpaStlhsjjnEms+jKyPVWD5afgEbqr2MK7ILGF58AzTRC9cSRL3/nP78vxyUvOjI3jH63oxs1bt++s3G3fu//g4aPO6tqeUZWmMKSKK32QEwOcSRhaZjkclBqIyDns54dv6/r+EWjDlNy18xIyQaaSFYwSG6RxZ5DmwqWDGfP4NR41yRHRZcjX60T7cbLRwymfKGt6V5V3NrJPLhXEzkzhdr0fd7pxP24CX4ZkAV20iMF4tfUlnShaCZCWcmLMKIlLmzmiLaMcfDutDJSEHpIpjAJKIsBkrmnd42dBmeBC6bCkxY367w5HhDFzkQdn88aLtVq8qjaqbPEqc0yWlQVJzy4qKo6twvU/4gnTQC2fByBUs/BWTGdEE2rDb7fPXZOLXn2+NoUJ3byD0KWGnaB8KEETq/RzlxI9FeTYh66naa+m/xmZXBoDXWcMhzDBPoN3f+laK5NL65LCGJOLQ7sMe5v9ZKu/9fFFd/vNYqAr6Al6itZRgl6ibfQeDdAQUfQd/US/0O/IRyfRafT1zBq1Fnseo3MRffsDvn34zA==</latexit> = ['(r1), . . . , '(rM )]T Truncation/discretization is necessary for constructing basis functions

Kernel regression for sound field esitmation Ø Problem to be
solved September 3, 2024 13 Kernel ridge regression with constraint that the interpolated function satisfies Helmholtz eq Solution space of Helmholtz eq § If is properly defined as reproducing kernel Hilbert space (RKHS), this problem has closed-form solution § Kernel regression can be regarded as infinite-dimensional basis expansion Good performance without truncation/discretization <latexit sha1_base64="CCNheoHdwk+3+XQETj9IXlSAYTs=">AAACtHicfZHbSsNAEIa38VzPeulNsAgiUhKR6qWoF96IClaFpshkO6mLewi7G7GGPIV4q8/l27iprXgeWPj459/ZmZ045czYIHiteCOjY+MTk1PV6ZnZufmFxaULozJNsUkVV/oqBoOcSWxaZjlepRpBxBwv49uDMn95h9owJc9tL8W2gK5kCaNgndSKBNgbQ3V+VFwv1IJ60A//J4QDqJFBnF4vVh6jjqKZQGkpB2NaYZDadg7aMsqxqEaZwRToLXSx5VCCQNPO+z0X/ppTOn6itDvS+n31840chDE9ETtnv8fvuVL8LdfKbLLbzplMM4uSvj+UZNy3yi8/wO8wjdTyngOgmrlefXoDGqh131T98kwsNsv62iTGTXOIbkqNx045SVGDVXojj0B3BdwXbuputFnSf0Ymh0ZHfxldESbYAxb5B/1pZXJoHZJbY/h9aT/hYqseNuqNs+3a3v5goZNkhaySdRKSHbJHjsgpaRJKFHkiz+TFa3iRRz18t3qVwZ1l8iU8+QacaNwN</latexit> H [Ueno+ 2018, 2021, Koyama+ ICASSP 2022 Tutorial]

Kernel regression for sound field esitmation Ø Unique solution with
closed-form for RKHS – Based on representer theorem, the solution is represented by weighted sum of reproducing kernel function : – Vector of is obtained by with September 3, 2024 14 Estimation is achieved by convoluting FIR filter in time domain <latexit sha1_base64="zt46LX9gSaZcNDUXQVGZZLwj3Zc=">AAADdnicjZHditNAFMcnjR9r/OrqpSCDZXVXSklEVm+ERb0QpLiC3S40pZ5MTrNDM5MwM1m2hjyFj+UT+CZeOmmbxXateGDgx//858w5c6I85dr4/k+n5V67fuPmzi3v9p279+63dx+c6KxQDAcsSzN1GoHGlEscGG5SPM0VgohSHEazd3V+eI5K80x+MfMcxwISyaecgbHSpP0jjET5sfLeUC+MMOGyjAQYxS8qL5xBnsN+bVDVJOjShg7oUxqyODO6hi2u/gENQy88b2xxA0ulzq3d7P9X/f5mfZTxZceTdsfv+YugVyFYQYes4niy63wP44wVAqVhKWg9CvzcjEtQhrMU7RcUGnNgM0hwZFGCQD0uF79e0T2rxHSaKXukoQv1zxslCK3nIrJO2+CZ3szV4t9yo8JMX49LLvPCoGTLh6ZFSk1G6xXSmCtkJp1bAKa47ZWyM1DAjF20t/ZMJLp1faWn2k7zHu2UCvtW+ZSjApOp52UIKhFwUdmpk7Bb07+MXDZGS9uMtggX/BtW5SVttXLZWBuyaww2l3YVTl70gsPe4eeXnaO3q4XukEfkCdknAXlFjsgHckwGhDmBM3S+OtD65T5299xnS2vLWd15SNbC9X8DK8EdQA==</latexit> K = 2 6 4 (r1, r1) · · · (r1, rM ) . . . ... . . . (rM , r1) · · · (rM , rM ) 3 7 5 : Gram matrix

Kernel regression for sound field esitmation Ø RKHS based on
plane wave expansion Ø Inner product and norm over using directional weighting September 3, 2024 15 How to design RKHS? Prior information on directions of high amplitude (e.g., source directions) can be incorporated <latexit sha1_base64="isRGdCdgAyvToZm2G6zcexSm7tk=">AAADfnicfVFbb9MwGHUWLqPcOnjkxaICdagrTTWN8YA0DR72AGIIuk2qS+S4X1LT2IlsByjGv4KfxhP/Bjdrq10YnxTp5Hznu/kkZc616fX+BGvhtes3bq7faty+c/fe/ebGgyNdVIrBgBV5oU4SqiHnEgaGmxxOSgVUJDkcJ9PX8/zxV1CaF/KTmZUwEjSTPOWMGk/Fzd9EUDPRTNkDh19hkkNqiMUkVZTZyNltUnKHCZcmtrU0SexHF/c9Z3g+Blu5NkmEJWCo28S1RAkL7rPdWv58cVPfmMosB7zSdmqofB9VZ9yqduxWIkww2edZ9nMOlvPm2+C3cb99dp8O/rZJFM8mhri42ep1e3XgyyBagBZaxGG8Efwi44JVAqRhOdV6GPVKM7JUGc78ag1SaSgpm9IMhh5KKkCPbP36Dj/xzBinhfKfNLhmz1ZYKrSeicQr67e+mJuT/8oNK5PujiyXZWVAstNBaZVjU+C5lXjMFTCTzzygTHG/K2YT6o0z3vDGuTGJ6NSvq1Ptr3kD/koF7zzzvgRFTaGeWUJVJuh356/OSGeO/ifkcin06Cqhb8IF/wHOrtCVUi6X0iXyNkYXTbsMjvrdaKe782G7tbe/MHQdPUKPURtF6AXaQwfoEA0QC14GcTAJeIjCp+FW+PxUuhYsah6icxHu/gUg4iX8</latexit> H = ( 1 4⇡ Z S 2 ˜ u(⌘)e jkh⌘,rid⌘ ˜ u 2 L2(S 2, w) )

Kernel regression for sound field esitmation Ø Kernel function for
based on von Mises‒Fisher distribution Ø When no prior information, i.e., uniform weight , September 3, 2024 16 How to design RKHS? with

Kernel regression for sound field esitmation Ø Experimental results using
real data from MeshRIR dataset – Reconstructing pulse signal from single loudspeaker w/ 18 mic September 3, 2024 17 True Proposed Gaussian kernel (Black dots indicate mic positions) Impulse response measurement system [Koyama+ 2021]

Neural networks for sound field estimation Ø Previous techniques based
on linear and kernel regressions – Prior information on source directions can be incorporated, but the estimator is manually designed and ﬁxed during estimation – Estimator adaptive to acoustic environment where the measurement is performed will have more ﬂexibility and high accuracy September 3, 2024 18 High representational power and adaptability to observations/datasets of neural networks will be useful for this purpose Input Output How to embed physical properties?

PINN for sound field estimation Ø Physics-informed neural network (PINN)
[Raissi+ 2019] – Based on implicit neural representation (or neural ﬁeld) to implicitly represent a continuous function by neural network – Loss function penalizing deviation from governing eq (or PDE loss/physics loss) is added to data loss/observation loss September 3, 2024 19 Input Output <latexit sha1_base64="upivb77T/u0SHof6imIve9fNe3o=">AAACznicfZFNbxMxEIad5asNXy0cuayIkAqqol2EChKXCjhwoQSJtJWyUTTrzG6s2t6VPVsRLIsrnPkhXMtP4d/gTRNEW8pIlh+98/pjZvJaCktJ8qsTXbl67fqNtfXuzVu379zd2Ly3b6vGcBzySlbmMAeLUmgckiCJh7VBULnEg/zodZs/OEZjRaU/0rzGsYJSi0JwoCBNNnrlVpYrZ/zLdstohgR+4jIFNDPK7e15/zi4kn6yiPgipEvosWUMJpud79m04o1CTVyCtaM0qWnswJDgEn03ayzWwI+gxFFADQrt2C2q8fGjoEzjojJhaYoX6t8nHChr5yoPzvaX9nyuFf+VGzVUvBg7oeuGUPPTh4pGxlTFbWviqTDISc4DADci/DXmMzDAKTSwe+aZXG0vOmQLG6p5g6FKg++C8r5GA1SZJy4DUyr45EPVZbbd0v+MQq+MgS4zhkuEEp/Ruz90qVXolXVFYYzp+aFdhP2n/XSnv/PhWW/31XKga+wBe8i2WMqes132lg3YkHH2jf1gJ+xnNIiOIx99ObVGneWZ++xMRF9/A92F5o0=</latexit> g(r; ✓NN) : Function represented by neural network

PINN for sound ﬁeld estimation Ø Physics-informed neural network (PINN)
[Raissi+ 2019] September 3, 2024 20 <latexit sha1_base64="/KSDW3J6TWfvx2FKerzDUKs9b8g=">AAADmXicjVLbbhMxEPV2uZRwaQqPfbGIkFJaomyECgJVKpeHUqkhiKatlE1WXsdJrNjele1FpI6f4A/4On6Cb8C72aJeCGIkS8cz54xnPBOnjCrdbP70VvwbN2/dXr1TuXvv/oO16vrDY5VkEpMuTlgiT2OkCKOCdDXVjJymkiAeM3IST9/l8ZMvRCqaiCM9S0mfo7GgI4qRdq6o+ivkVFBOz0hkwpibUE+IRtZdONITyU27ba0tLhgxc3Ah0vmQx+pLVZvw1S4MVcYjw3cDOziEcxVx+AyOC420EX+9XDwftOBWKRe5vA3n9VCgmKFFpdLaQWtrOmht/l9CmGeMqrVmo1kYvA6CEtRAaZ1o3fsRDhOccSI0ZkipXtBMdd8gqSlmxFbCTJEU4Skak56DAnGi+qYYjIVPnGcIR4l0R2hYeC8qDOJKzXjsmHmp6mosd/4t1sv06GXfUJFmmgi8eGiUMagTmE8ZDqkkWLOZAwhL6mqFeIIkwtrtQuXSMzHfLr5JjZTr5j1xXUpy6DwfUyKRTuRTEyI55uirdV2Pw+0c/YtIxTnRoWVEl6TYOmv+oKXUckEd9Ry5MQZXh3YdHLcawU5j59Pz2t7bcqCrYAM8BnUQgBdgD+yDDugC7B15Z94377u/4b/x9/2DBXXFKzWPwCXzP/8GU9syEg==</latexit> minimize ✓NN JPINN(✓NN) := M X m=1 |sm g(rm; ✓NN)|2 + N X n=1 |(r2 r + k2)g(rm; ✓NN)|2 <latexit sha1_base64="zCg6p69HjmrJB4GrY+2kRUfUKOo=">AAACxnicfZHdShtBFMcnq21t+mGsl94shkIpEnZFrJdivZBCqQWjQjaEs5OzcXBmdpk5axuHhfYZfIjeto/Tt+nsmkjV2gMDv/mf/3ycc9JCCktR9LsVLCw+evxk6Wn72fMXL5c7K6+ObV4ajn2ey9ycpmBRCo19EiTxtDAIKpV4kp6/r/MnF2isyPURTQscKphokQkO5KVRZy1RQGccpPtQjVyzMcqNgaCqRp1u1IuaCO9DPIMum8XhaKV1lYxzXirUxCVYO4ijgoYODAkusWonpcUC+DlMcOBRg0I7dE0VVfjaK+Mwy41fmsJG/fuEA2XtVKXeWX/T3s3V4r9yg5KynaETuigJNb9+KCtlSHlYtyQcC4Oc5NQDcCP8X0N+BgY4+ca1bz2Tqo2mRTazvpp99FUa/OiVTwUaoNy8dQmYiYKvla96kmzU9D+j0HOjp4eM/hKhxCVW7oYetAo9t87JjzG+O7T7cLzZi7d725+3urt7s4EusTW2zt6wmL1ju+yAHbI+4+w7+8F+sl/BQaCDMvhybQ1aszOr7FYE3/4AShLjlg==</latexit> Jdata <latexit sha1_base64="w7F/E1I+85w8BY154v47c18Ny5s=">AAACxXicfZHdahNBFMcn61eNH031Tm8WgyBSwm4ptZdFKy0FMYJpC9kQzk7OpkNnZpeZs6VxWPQdfIfe6uv0bTq7TcS21gMDv/mf/3ycc9JCCktRdN4K7ty9d//B0sP2o8dPni53Vp7t27w0HAc8l7k5TMGiFBoHJEjiYWEQVCrxID3+UOcPTtBYkeuvNCtwpGCqRSY4kJfGnReJAjriIN1eNXbNxijX3/5YVeNON+pFTYQ3IZ5Dl82jP15p/UwmOS8VauISrB3GUUEjB4YEl1i1k9JiAfwYpjj0qEGhHbmmiCp87ZVJmOXGL01ho/59woGydqZS76x/aa/navFfuWFJ2ebICV2UhJpfPpSVMqQ8rDsSToRBTnLmAbgR/q8hPwIDnHzf2leeSdVq0yGbWV/NNvoqDX7yyucCDVBu3roEzFTBaeWrniarNf3PKPTC6Ok2o79EKPENK/eHbrUKvbAuyI8xvj60m7C/1os3ehtf1rtb7+cDXWIv2Sv2hsXsHdtiu6zPBoyzH+yM/WK/g51ABRScXFqD1vzMc3Ylgu8XYYXiyw==</latexit> JPDE Ground truth Reconstruction from 33 ch PINN NN [Pezzoli+ 2023]

Feedforward neural network for sound field estimation Ø Drawbacks of
PINN – Basically, single array observation is used for training and inference, i.e., training data set is not used – Estimate does not strictly satisfy Helmholtz/wave eq because of penalization by PDE loss September 3, 2024 21 How can physical properties be embedded in neural networks that have discrete output values? Input Output

Feedforward neural network for sound field estimation Ø Estimating expansion
coefs of basis expansion using neural networks – Train a model estimating expansion coefs of basis expansion – Continuous function can be reconstructed by using estimated expansion coefs – Can be regarded as physics-constrained neural network (PCNN) [Karakonstantis+ 2023, Lobato+ 2024] Ø Approximate PDE loss – Because of discrete output values, computation of PDE loss is not straightforward – Approximate PDE loss can be computed by finite difference or interpolation – In [Shigemi+ 2022], physics-informed convolutional neural network (PICNN) using bi- cubic interpolation is proposed September 3, 2024 22

Neural kernel for sound ﬁeld estimation Ø Kernel function with
constrainst of Helmholtz eq is optimized to acoustic environment with the aid of neural networks [Ribeiro+ 2023] – Superposition of two kernel functions – Directed kernel: direct source and early reflections – Residual kernel: late reverberations and residual components September 3, 2024 23 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="0rmx5Ei2B3tRv8KkicAKVISG2OQ=">AAAC4XicfZHNThsxEMedLW1p+hXKsReLqFLVomi3qgKXSqjl0EsFSASQslE068wGK7bXsr2IdLUP0FsFR3iaXssL8DZ481EVUjqSpZ/+87fHM5Nowa0Lw+ta8GDp4aPHy0/qT589f/GysfLqwGa5YdhhmcjMUQIWBVfYcdwJPNIGQSYCD5PRlyp/eILG8kztu7HGnoSh4iln4LzUb7TjEWgN9BOdQr+IJbhjI4sBN2VJ3y/oBm1Z9hvNsBVOgi5CNIMmmcVuf6V2Hg8ylktUjgmwthuF2vUKMI4zgWU9zi1qYCMYYtejAom2V0waLOkbrwxomhl/lKMT9e8bBUhrxzLxzuqX9m6uEv+V6+Yu3ewVXOncoWLTQmkuqMtoNS3qh4DMibEHYIb7v1J2DAaY8zOt3yqTyPXJhGxqfTfb6Ls0+M0rOxoNuMy8K2IwQwmnpe96GK9X9D8jV3Ojp/uM/hEu+Xcsiz90r5WruXVOfo3R3aUtwsGHVtRutfc+Nrc+zxa6TF6TNfKWRGSDbJGvZJd0CCOX5Bf5Ta4CFvwIfgZnU2tQm91ZJbciuLgBJGTucg==</latexit>  = dir + res Directed kernel Residual kernel

Neural kernel for sound field estimation Ø Directed kernel –
Directional weighting with weighted sum of (sparse) von Mises—Fisher distribution [Horiuchi+ 2021] September 3, 2024 24 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="I2jQ2Fgmjq+z5rZFxQfy+e9Ar9o=">AAADQ3icfZHNbtQwEMed8FWWj27hyMViQdqiZZWgqkVClSrKgQtQJLattNlGjneSWo2dyHaAxfJTcOGFeAiegRviCsJJdle0pYwU6af//GecmUnKnCkdBN88/9LlK1evrVzv3Lh56/Zqd+3OvioqSWFEi7yQhwlRkDMBI810DoelBMKTHA6Sk906f/AepGKFeKdnJUw4yQRLGSXaSXH3y4fYRJzoY8nNlElr+1HCTQSa2Ge4waTmQcsZ4ZzYdbyNI1Xx2Ijt0B69xq0eCxylktBlP7BHbXmdyYnIcsDL7m3Hqa1zsslZa3b7c/+6jbu9YBg0gc9DOIcemsdevOZ9jqYFrTgITXOi1DgMSj0xRGpGXfdOVCkoCT0hGYwdCsJBTUyzQosfOmWK00K6T2jcqH9XGMKVmvHEOevp1NlcLf4rN650+nRimCgrDYK2D6VVjnWB63tgt3SgOp85IFQy96+YHhO3Ru2u1jn1TMIHzWZVqtw0L8BNKeGVU96UIIku5CMTEZlx8tG6qbNoUNP/jEwsjI4uMromjLNPYM2SLrQysbAuyJ0xPHu087D/ZBhuDjffbvR2ns8PuoLuofuoj0K0hXbQS7SHRoii394D77E39L/63/0f/s/W6nvzmrvoVPi//gCenhPw</latexit> wdir(⌘; , ) = N X n=1 n e n h⌘,dn i C( n) <latexit sha1_base64="wce9c2fonwWBBzEGuOJL2nk9Ius=">AAACwHicfZFbSxtBFMcnq201vUXFJ18WQ8EWCbulaF8EUR98ERUaFbIhnJ2c3QzOZZ2ZLcZ1P4WfwFf9RH4bZ2Mi9dYDAz/+5z8z5xJnnBkbBHc1b2r63fsPM7P1j58+f/namJs/MirXFNtUcaVPYjDImcS2ZZbjSaYRRMzxOD7drvLHf1EbpuQfO8ywKyCVLGEUrJN6jcWV6DKKRRGlIASU0WUv3Ai/9xrNoBWMwn8J4RiaZBwHvbnaVdRXNBcoLeVgTCcMMtstQFtGOZb1KDeYAT2FFDsOJQg03WJUf+l/c0rfT5R2R1p/pP57owBhzFDEzinADszzXCW+luvkNvndLZjMcouSPnyU5Ny3yq+G4feZRmr50AFQzVytPh2ABmrdyOpPvonFavW+Nolx3eyg61LjnlP2M9Rglf5RRKBTAeel6zqNViv6n5HJidHRW0b3CBPsAsvikd60MjmxTsitMXy+tJdw9LMVrrXWDn81N7fGC50hS2SZrJCQrJNNsksOSJtQUpBrckNuvS1v4Cnv7MHq1cZ3FsiT8C7uAZQE35U=</latexit> (k k1 = 1) <latexit sha1_base64="44H961efgdVBY4JkEGr9+jx5ReM=">AAADfHiclZFdb9MwFIadlo9RPtaNS24sClILXZVUqONm0sS44AYY0rpNqrvIcZ3Ua+xktoMoln8Fv4xL/gzCSVPENobEkSI9Ouc95+T4jfKUKe37P7xG89btO3c37rXuP3j4aLO9tX2sskISOiZZmsnTCCuaMkHHmumUnuaSYh6l9CRaHJT1k89UKpaJI73M6ZTjRLCYEaxdKmx/Rwuc5zg0iGM9l9zMmLS2iyJupA2Dfg3DHtyDSBU8NGIvsGcfIEow5zgUsIViiYk5D32U0lh3nexCagO764nnFkVUY1iOQg4s3IGLeq4JhrZ3tlquYnNk7X/0QYskS+a6Z81Bt9KGomfDdscf+FXA6xDU0AF1HIZb3jc0y0jBqdAkxUpNAj/XU4OlZiSltoUKRXNMFjihE4cCc6qmpnp7C5+7zAzGmXSf0LDK/tlhMFdqySOnrI68WiuTf6tNCh2/nhom8kJTQVaL4iKFOoOlkdAZRYlOlw4wkcz9KyRz7KzQzu7WpTUR71dvqmLlrnlL3ZWSvneZjzmVWGfyhUFYJhx/se7qBPVL+peQibXQ0U1CN4Rx9pVa85tulDKxlq7J2RhcNe06HA8HwWgw+vSqs/+mNnQDPAFPQRcEYBfsg3fgEIwB8Xa9qRd7SeNn81nzZXNnJW14dc9jcCmao1839ST1</latexit> dir(r1, r2) = N X n=1 n j0 ⇣p (j ⌘ kr12)T(j ⌘ kr12) ⌘ C( n) Sparsity constraint Normalization constant

Neural kernel for sound ﬁeld estimation Ø Residual kernel –
Directional weighting with implicit neural representation September 3, 2024 25 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="s4P1x3nuyJ2mvTCbZ2ZsvG/WxFE=">AAAC+XicfZHdahNBFMcn61eNX6leejMYhCol7BaphSIU9cIbYwXTFrIhnJ2cTYbuzC4zZ9U47FP4BN6Jt/oyeqsP4uw2EdsaDwz85n/+83HOSYpMWgrD763gwsVLl6+sXW1fu37j5q3O+u0Dm5dG4EDkWW6OErCYSY0DkpThUWEQVJLhYXL8rM4fvkVjZa7f0LzAkYKplqkUQF4ad/rvxi5WQDOjnEFbVRtxolyMBNUub5Bm9eYBf8KXvn5/pWvc6Ya9sAl+HqIFdNki9sfrrY/xJBelQk0iA2uHUVjQyIEhKTKs2nFpsQBxDFMcetSg0I5cU3jF73tlwtPc+KWJN+rfJxwoa+cq8c767/Zsrhb/lRuWlO6MnNRFSajFyUNpmXHKed1FPpEGBWVzDyCM9H/lYgYGBPlet089k6jNpm82tb6a5+irNPjSK68KNEC5eehiMFMF7ytf9TTerOl/RqmXRk+rjP4SqeQHrNwfWmmVemldkh9jdHZo5+Fgqxdt97ZfP+ruPV0MdI3dZffYBovYY7bHXrB9NmCCfWM/2E/2K3DBp+Bz8OXEGrQWZ+6wUxF8/Q016fgt</latexit> wres(⌘; ✓) = NN(⌘; ✓) <latexit sha1_base64="dexZadk2Poc298GGiqVC2ayTShI=">AAADX3icfZHfihMxFMYzrbpr1bWrV+pNsAi7UktnkVUQoagXKogr2t2FppYz6WkbO8kMSUatQ57CV/FlvPRNzEynxf3ngYHffOc7OTk5URoLY7vd30GtfunylY3Nq41r129s3Wxu3zo0SaY59nkSJ/o4AoOxUNi3wsZ4nGoEGcV4FM1fFvmjr6iNSNQnu0hxKGGqxERwsF4aNX+xOaQpjHImwc60zDUa53ZY5MmNwnYFe7v0OWVC2coYRflHrzr67bxKhhbcs5LsrOBdujKh+5w/Wv18cSwGNY2RrqvadNmRMl1m3Lpy7NamUbPV7XTLoGchrKBFqjgYbQc/2TjhmURleQzGDMJuaoc5aCu4b9JgmcEU+BymOPCoQKIZ5uXzOvrAK2M6SbT/lKWl+m9FDtKYhYy8s7irOZ0rxPNyg8xOng5zodLMouLLRpMspjahxa7oWGjkNl54AK6FvyvlM9DArd9o40SbSLbLdzIT46d5hX5Kje+88j5FDTbRD3MGeirhu/NTT1m7oP8ZhVoZPV1k9IcIKX6gy9d0oVWolXVFfo3h6aWdhcO9Trjf2f/wuNV7US10k9wj98kOCckT0iOvyQHpEx7cDXrBm+Bt7U99o75Vby6ttaCquU1ORP3OX+2/HOQ=</latexit> res(r1, r2) = Z S 2 wres(⌘; ✓)e jh⌘,rid⌘ Computed by numerical integration : Implicit neural representation

Neural kernel for sound field estimation Ø Again, (positive-deﬁnite) kernel
function is the sum of directed and residual kernels – Hyperparameters are jointly optimized by a steepest-descent-based algorithm – Physics-constraint is still preserved – Estimation process is still linear operation in freq domain based on kernel ridge regression September 3, 2024 26 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="0rmx5Ei2B3tRv8KkicAKVISG2OQ=">AAAC4XicfZHNThsxEMedLW1p+hXKsReLqFLVomi3qgKXSqjl0EsFSASQslE068wGK7bXsr2IdLUP0FsFR3iaXssL8DZ481EVUjqSpZ/+87fHM5Nowa0Lw+ta8GDp4aPHy0/qT589f/GysfLqwGa5YdhhmcjMUQIWBVfYcdwJPNIGQSYCD5PRlyp/eILG8kztu7HGnoSh4iln4LzUb7TjEWgN9BOdQr+IJbhjI4sBN2VJ3y/oBm1Z9hvNsBVOgi5CNIMmmcVuf6V2Hg8ylktUjgmwthuF2vUKMI4zgWU9zi1qYCMYYtejAom2V0waLOkbrwxomhl/lKMT9e8bBUhrxzLxzuqX9m6uEv+V6+Yu3ewVXOncoWLTQmkuqMtoNS3qh4DMibEHYIb7v1J2DAaY8zOt3yqTyPXJhGxqfTfb6Ls0+M0rOxoNuMy8K2IwQwmnpe96GK9X9D8jV3Ojp/uM/hEu+Xcsiz90r5WruXVOfo3R3aUtwsGHVtRutfc+Nrc+zxa6TF6TNfKWRGSDbJGvZJd0CCOX5Bf5Ta4CFvwIfgZnU2tQm91ZJbciuLgBJGTucg==</latexit>  = dir + res Directed kernel Residual kernel <latexit sha1_base64="LfIE5/umsVmKZKr2rKhL6DRhj+4=">AAACz3icfZFNbxMxEIad5auEj6Zw5GIRkBCKot2qKhwr6IELaiuRtlI2imad2Y1V27uyZ4GwWtQrXPkjXOk/6b+pN00q2lJGsvT4nXdsjycplHQUhqet4NbtO3fvrdxvP3j46PFqZ+3JvstLK3AgcpXbwwQcKmlwQJIUHhYWQScKD5Kj903+4DNaJ3PziWYFjjRkRqZSAHlp3HkRJ7qKEySoe3zOGWh9saFpkxl3umE/nAe/DtECumwRu+O11q94kotSoyGhwLlhFBY0qsCSFArrdlw6LEAcQYZDjwY0ulE1b6fmL70y4Wlu/TLE5+rfFRVo52Y68U4NNHVXc434r9ywpPTtqJKmKAmNOL8oLRWnnDd/wyfSoiA18wDCSv9WLqZgQZD/wfalaxLda863LnW+m230XVr86JWdAi1Qbl9XMdhMw9fad53FvYb+Z5RmafR0k9EfIrX8hnV1QTdapVlal+THGF0d2nXYX+9Hm/3NvY3u1rvFQFfYM/acvWIRe8O22Ae2ywZMsJ/sN/vDToK94EvwPTg+twatRc1TdimCH2e3guZ0</latexit> , , ✓

Neural kernel for sound field estimation Ø Numerical experiment: T60:
400 ms, # mics: 41, spherical shell array September 3, 2024 27 Ground truth (600 Hz) NN PINN Adaptive kernel NMSE: -6.8 dB NMSE: -16.3 dB NMSE: -24.8dB

Overview article for PIML-based sound ﬁeld estimation Ø Physics-Informed Machine
Learning For Sound Field Estimation – To appear in IEEE Signal Processing Magazine, but preprint is available below September 3, 2024 28 https://arxiv.org/abs/2408.14731

Application to Binaural Reproduction September 3, 2024 29 Conversion into
binaural sounds Ø Binaural reproduction in real world is difficult, compared to binaural synthesis in VR space Ø Binaural reproduction from recordings of multiple small arrays instead of single spherical array Ø Broad listening area by using flexible and scalable recording system Binaural reproduction from mic array recordings for VR audio Recording Reproduction [Iijima+ JASA 2021]

Application to Binaural Reproduction Ø Recording system using multiple Ambisonic
mics and 360-degree cameras September 3, 2024 30 Small mic arrays (Ambisonic mics) 360-degree cameras Demo Proposed Single array [Iijima+ IEEE WASPAA 2021 (demo)] Error distribution

September 3, 2024 31 Headphones Recommended

SOUND FIELD CONTROL September 3, 2024 32

Sound field control problem Ø Optimization problem to be solved
September 3, 2024 33 Difficult to solve owing to regional integration Goal: Synthesizing desired sound field inside with secondary sources (loudspeakers) <latexit sha1_base64="SbdBLMPsjw6ciQkjzJuiJJhjGvk=">AAACF3icbVDLSsNAFJ3UV62vqksXBotQQUoiRV0W3bisYB/QlDCZ3LZDZyZhZiKUkKUf4Te41bU7cevSpX/i9LGwrQcuHM65l3vvCWJGlXacbyu3srq2vpHfLGxt7+zuFfcPmipKJIEGiVgk2wFWwKiAhqaaQTuWgHnAoBUMb8d+6xGkopF40KMYuhz3Be1RgrWR/OJx4qcex3ogeRqCyrKyF/BUZudexKGPz/xiyak4E9jLxJ2REpqh7hd/vDAiCQehCcNKdVwn1t0US00Jg6zgJQpiTIa4Dx1DBeaguunkkcw+NUpo9yJpSmh7ov6dSDFXasQD0zm+WS16Y/E/r5Po3nU3pSJONAgyXdRLmK0je5yKHVIJRLORIZhIam61yQBLTLTJbm5LwDOTibuYwDJpXlTcy0r1vlqq3czSyaMjdILKyEVXqIbuUB01EEFP6AW9ojfr2Xq3PqzPaWvOms0cojlYX7/bXqDc</latexit> udes(r, !) <latexit sha1_base64="OlngG7bMYm0AnUElZFcGXeuuUPg=">AAAB/HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E1DJoY2cE8wHJEfY2k2TN7t2xuyeEI/4GW63txNb/Yuk/cZNcYRIfDDzem2FmXhALro3rfju5ldW19Y38ZmFre2d3r7h/UNdRohjWWCQi1QyoRsFDrBluBDZjhVQGAhvB8GbiN55QaR6FD2YUoy9pP+Q9zqixUr19J7FPO8WSW3anIMvEy0gJMlQ7xZ92N2KJxNAwQbVueW5s/JQqw5nAcaGdaIwpG9I+tiwNqUTtp9Nrx+TEKl3Si5St0JCp+ncipVLrkQxsp6RmoBe9ifif10pM78pPeRgnBkM2W9RLBDERmbxOulwhM2JkCWWK21sJG1BFmbEBzW0J5Nhm4i0msEzqZ2Xvonx+f16qXGfp5OEIjuEUPLiECtxCFWrA4BFe4BXenGfn3flwPmetOSebOYQ5OF+/In2Vmg==</latexit> ⌦ <latexit sha1_base64="VJ5RMQ2GKmQZdUQJz96dZvLRnxA=">AAAB93icbVA9SwNBEN2LXzF+RS1tFoNgFe4kqGXQxsIiAfMByRH2NnPJkt29Y3dPOI78Alut7cTWn2PpP3GTXGESHww83pthZl4Qc6aN6347hY3Nre2d4m5pb//g8Kh8fNLWUaIotGjEI9UNiAbOJLQMMxy6sQIiAg6dYHI/8zvPoDSL5JNJY/AFGUkWMkqMlZqPg3LFrbpz4HXi5aSCcjQG5Z/+MKKJAGkoJ1r3PDc2fkaUYZTDtNRPNMSETsgIepZKIkD72fzQKb6wyhCHkbIlDZ6rfycyIrRORWA7BTFjverNxP+8XmLCWz9jMk4MSLpYFCYcmwjPvsZDpoAanlpCqGL2VkzHRBFqbDZLWwIxtZl4qwmsk/ZV1buu1pq1Sv0uT6eIztA5ukQeukF19IAaqIUoAvSCXtGbkzrvzofzuWgtOPnMKVqC8/ULSPOTbw==</latexit> L : Target region Secondary source Synthesized sound field <latexit sha1_base64="jXBfTdmzlzCxe3eJ2RCk10ZxZxw=">AAACo3icfZHbSgMxEIbT9VzPeunNYhFEpOyKqJeiXggiHqtCdymz6XQbTLJLkhXrsi8geKvP5tuY1lY8DwQ+/vmTmclEKWfaeN5ryRkaHhkdG58oT05Nz8zOzS9c6SRTFGs04Ym6iUAjZxJrhhmON6lCEBHH6+h2v5u/vkOlWSIvTSfFUEAsWYtRMFa6aDZ4Y67iVb1euD/B70OF9OO0MV96DJoJzQRKQzloXfe91IQ5KMMox6IcZBpToLcQY92iBIE6zHu9Fu6KVZpuK1H2SOP21M83chBad0RknQJMW3/PdcXfcvXMtHbCnMk0Myjpe6FWxl2TuN3B3SZTSA3vWACqmO3VpW1QQI39ni9VIlGUy8EB2uEUHttCJykqMIlaywNQsYD7wg4bB+td+s/I5MBo6S+jfYQJ9oBF/kF/WpkcWAdkt+d/39VPuNqo+lvVzbPNyu5ef4/jZIksk1Xik22ySw7JKakRSmLyRJ7Ji7PiHDnnzuW71Sn17yySL+GEb9ci1JI=</latexit> dl <latexit sha1_base64="2143p1RAQMlzka1H5YkQBHUMHYo=">AAACq3icfZFLb9NAEMc3ppQS6AuOXKxGSKVqIxtVLceo7YELIqhNWxFH0XgzdlbZh7W7RgTLX4B7r/C5+DasE6dqWspIK/0085+dV5xxZmwQ/Gl4T1aerj5be9588XJ9Y3Nr+9WlUbmm2KOKK30dg0HOJPYssxyvM40gYo5X8eS0il99Q22Ykhd2muFAQCpZwihY5+qnQ74bxaLQ5bvhVitoBzPzH0JYQ4vU1h1uN35GI0VzgdJSDsb0wyCzgwK0ZZRj2YxygxnQCaTYdyhBoBkUs55L/63zjPxEafek9WfeuxkFCGOmInZKAXZs7scq579i/dwmHwYFk1luUdJ5oSTnvlV+tQB/xDRSy6cOgGrmevXpGDRQ69a0VCUWZbMZnaEbTuMnV+hzhhqs0ntFBDoV8L10w6bRfkX/EzK5EDp6TOg+YYL9wLK4pUelTC6kC3LXC+/f6iFcvm+HR+3DL4etzkl9xzXyhuyQXRKSY9IhH0mX9AglityQX+S3d+Cde1+9aC71GnXOa7JkHv4Fm6XXyw==</latexit> gl(r) • : Driving signal of th secondary sources • : Transfer function of th secondary source <latexit sha1_base64="bKgRyukOmCpfZztJEARaEMlc1nM=">AAACoXicfZFNSwMxEIbT9bt+Vo9eFosgImVXRD0W9aAHsYrVQreU2XRag0l2SbJiXfYP6FV/nP/GtLZi1ToQeHjnTWYmE8acaeN57zlnYnJqemZ2Lj+/sLi0vFJYvdFRoihWacQjVQtBI2cSq4YZjrVYIYiQ4214f9zL3z6g0iyS16YbY0NAR7I2o2CsdMmbK0Wv5PXD/Q3+AIpkEJVmIfcctCKaCJSGctC67nuxaaSgDKMcs3yQaIyB3kMH6xYlCNSNtN9p5m5apeW2I2WPNG5f/X4jBaF1V4TWKcDc6Z+5nvhXrp6Y9mEjZTJODEr6WaidcNdEbm9st8UUUsO7FoAqZnt16R0ooMZ+zkiVUGT5fHCCdjiF57bQRYwKTKS20wBUR8BjZoftBDs9+s/I5NBoaZzRPsIEe8Is/aKxViaH1iHZ7fk/d/UbbnZL/n5p73KvWD4a7HGWrJMNskV8ckDK5JRUSJVQguSFvJI3p+icORXn6tPq5AZ31shIOPUP1irTuw==</latexit> l <latexit sha1_base64="bKgRyukOmCpfZztJEARaEMlc1nM=">AAACoXicfZFNSwMxEIbT9bt+Vo9eFosgImVXRD0W9aAHsYrVQreU2XRag0l2SbJiXfYP6FV/nP/GtLZi1ToQeHjnTWYmE8acaeN57zlnYnJqemZ2Lj+/sLi0vFJYvdFRoihWacQjVQtBI2cSq4YZjrVYIYiQ4214f9zL3z6g0iyS16YbY0NAR7I2o2CsdMmbK0Wv5PXD/Q3+AIpkEJVmIfcctCKaCJSGctC67nuxaaSgDKMcs3yQaIyB3kMH6xYlCNSNtN9p5m5apeW2I2WPNG5f/X4jBaF1V4TWKcDc6Z+5nvhXrp6Y9mEjZTJODEr6WaidcNdEbm9st8UUUsO7FoAqZnt16R0ooMZ+zkiVUGT5fHCCdjiF57bQRYwKTKS20wBUR8BjZoftBDs9+s/I5NBoaZzRPsIEe8Is/aKxViaH1iHZ7fk/d/UbbnZL/n5p73KvWD4a7HGWrJMNskV8ckDK5JRUSJVQguSFvJI3p+icORXn6tPq5AZ31shIOPUP1irTuw==</latexit> l

Pressure matching Ø Discretize target region into control points Ø
Optimization problem for pressure matching becomes simple least-squares problem September 3, 2024 34 <latexit sha1_base64="V9SycBRbehDy9zsTAA8X3hDIM1c=">AAACpnicfZFLSwMxEMfT9V2f1aOXxSKISNkVUY9FPXjxBbYVukVn0+kaTbJLkhXrsl/Ak1f9Zn4b09qK74HAj//8k5nJhAln2njea8EZGR0bn5icKk7PzM7NL5QW6zpOFcUajXmsLkLQyJnEmmGG40WiEETIsRHe7vfyjTtUmsXy3HQTbAmIJOswCsZK9eBEYASXC2Wv4vXD/Qn+AMpkEKeXpcJj0I5pKlAaykHrpu8lppWBMoxyzItBqjEBegsRNi1KEKhbWb/d3F21StvtxMoeady++vlGBkLrrgitU4C51t9zPfG3XDM1nd1WxmSSGpT0vVAn5a6J3d7sbpsppIZ3LQBVzPbq0mtQQI39oS9VQpEXi8EB2uEUHtlCJwkqMLFazwJQkYD73A4bBRs9+s/I5NBo6S+jfYQJ9oB59kF/WpkcWodkt+d/39VPqG9W/O3K1tlWubo32OMkWSYrZI34ZIdUySE5JTVCyQ15Is/kxVlzjp2a03i3OoXBnSXyJZyrN7rP1cY=</latexit> ⌦ <latexit sha1_base64="q0uECYVB1NlCDNpVOene+1Qk19c=">AAACq3icfZFLTxsxEMedpVBIeXPkYhFVggqi3QoBR0Q5cKAF1AZQsxGadSYbCz9WtheRrvYLcOcKn6vfpk5IKt4jWfpp5j+eV5IJbl0Y/q0EYx/GJz5OTlU/Tc/Mzs0vLJ5anRuGDaaFNucJWBRcYcNxJ/A8MwgyEXiWXH7rx8+u0Fiu1S/Xy7AlIVW8wxk472r+oDFdjVOkh2sX87WwHg6MvoRoCDUytOOLhcpN3NYsl6gcE2BtMwoz1yrAOM4EltU4t5gBu4QUmx4VSLStYtBzST97T5t2tPFPOTrwPs4oQFrbk4lXSnBd+zzWd74Wa+aus9MquMpyh4o9FOrkgjpN+wugbW6QOdHzAMxw3ytlXTDAnF/TkyqJLKvVeB/9cAa/+0JHGRpw2nwpYjCphOvSD5vG6316T8jVSOjpLaH/hEv+B8viP70p5WokHZG/XvT8Vi/h9Gs92qpvnmzWdveGd5wky2SFrJKIbJNdckCOSYMwosktuSP3wUbwM/gdxA/SoDLMWSJPLMB/tKXWgg==</latexit> N ( L) <latexit sha1_base64="Sfob4tqImS+gkH7eBMlgYAHh4Uo=">AAADDnicfZJNb9QwEIad8FWWj27hyMVihYT4WCVVRXusKBIcQBSJbSutd1eOd5K1ajuR7SCWkD/AiV/DDXHlzI1/wpFJmiLaUkaK9OqdZzweT5JCSeej6GcQXrh46fKVlau9a9dv3Fztr93ac3lpBYxErnJ7kHAHShoYeekVHBQWuE4U7CeHO01+/x1YJ3Pz1i8LmGieGZlKwT1as75iWhqp5QeYVSzR1bymTBrKNPeLJKl26ulLdBSknn2kDfAcicetKutp1XIWy8DVyFmZLRCcrtOHlIHntCtqTkV31h9Ew6gNelbEnRiQLnZna8EnNs9FqcF4obhz4zgq/KTi1kuhoO6x0kHBxSHPYIzScA1uUrXPUtN76Mxpmlv8jKet+3dFxbVzS50g2YzhTuca81+5cenTrUklTVF6MOKoUVoq6nPavDGdSwvCqyUKLqzEu1Kx4JYLj5s40SXRda/HngEOZ+EVNnpdgOU+tw8qxm2m+fsah83Yo0b9D5TmGER1HoiHtKuuqz/qXLT7KxA9Vri9+PSuzoq99WH8ZLjxZmOw/bTb4wq5Q+6S+yQmm2SbvCC7ZEQE+UF+BUEQhp/DL+HX8NsRGgZdzW1yIsLvvwHgg/35</latexit> minimize d2CL Gd udes 2 + ⌘kdk2 <latexit sha1_base64="i9WfaOjLrhWO59r+nVt9MC7/SLM=">AAADEHicfZHLbhMxFIY90wIlXJrCko1FhFRu0QyqgA1SBUiUBaJIpK0Up5HHOTOxantGtgcRLL8AS56GHWLLmg2vwgrPJEH0xpEsff7Pf3xsn6wS3Ngk+RXFK6sXLl5au9y5cvXa9fXuxo09U9aawYCVotQHGTUguIKB5VbAQaWBykzAfnb0osnvfwBteKne21kFI0kLxXPOqA3SuKtIJt3E42eYCMjtJm72r/yhI5LaqcndjvdzCd/HBCxtDa89JpoXU3v30D1M/VlFrVYvNR2agPF+3O0l/aQNfBrSBfTQInbHG9FnMilZLUFZJqgxwzSp7MhRbTkT4DukNlBRdkQLGAZUVIIZufZjPL4TlAnOSx2WsrhV/61wVBozk1lwtjc/mWvEs3LD2uZPR46rqrag2LxRXgtsS9z8Mp5wDcyKWQDKNA93xWxKNWU2zOJYl0z6Toe8hPA4DW9Co7cVaGpLfc8RqgtJP/rw2II8aOh/Rq6WxkDnGcMhXPJP4N1fOtfK1dK6pDC99OSsTsPeo376uL/1bqu3/XwxxzV0C91GmyhFT9A22kG7aIAY+ol+RyvRavwl/hp/i7/PrXG0qLmJjkX84w9/cP+K</latexit> d = GHG + ⌘I 1 GHudes Transfer function matrix Driving signal vector Desired pressure vector Regularization term Closed-form solution is obtained as J Simple implementation L Fine discretization of is necessary <latexit sha1_base64="V9SycBRbehDy9zsTAA8X3hDIM1c=">AAACpnicfZFLSwMxEMfT9V2f1aOXxSKISNkVUY9FPXjxBbYVukVn0+kaTbJLkhXrsl/Ak1f9Zn4b09qK74HAj//8k5nJhAln2njea8EZGR0bn5icKk7PzM7NL5QW6zpOFcUajXmsLkLQyJnEmmGG40WiEETIsRHe7vfyjTtUmsXy3HQTbAmIJOswCsZK9eBEYASXC2Wv4vXD/Qn+AMpkEKeXpcJj0I5pKlAaykHrpu8lppWBMoxyzItBqjEBegsRNi1KEKhbWb/d3F21StvtxMoeady++vlGBkLrrgitU4C51t9zPfG3XDM1nd1WxmSSGpT0vVAn5a6J3d7sbpsppIZ3LQBVzPbq0mtQQI39oS9VQpEXi8EB2uEUHtlCJwkqMLFazwJQkYD73A4bBRs9+s/I5NBo6S+jfYQJ9oB59kF/WpkcWodkt+d/39VPqG9W/O3K1tlWubo32OMkWSYrZI34ZIdUySE5JTVCyQ15Is/kxVlzjp2a03i3OoXBnSXyJZyrN7rP1cY=</latexit> ⌦ : Target region Secondary source Control points

Weighted Pressure Matching Ø Original cost function is approximated as
Ø Driving signals are obtained as weighted least squares solution September 3, 2024 35 Pressure matching for continuous region based on kernel regression of sound field <latexit sha1_base64="wBoAciZ2FK9Swr355ix7ob1QUvE=">AAADEnicfZHLbtNAFIYn5lbCLYUlmxERUkEosqsK2CBVLQs2qEVqmoo4RMeTY3dUz9iaGVdNR/MWPADPwQrElhfgBdjCIzB2EkR64UiWP/3nn9t/kjLn2oThj1Zw5eq16zdWbrZv3b5z915n9f6+LirFsM+KvFAHCWjMucS+4SbHg1IhiCTHQXK0XfcHx6g0L+SemZY4EpBJnnIGxkvjzvs4EXbg6Csac2nGNt4RmIGjtXzq1uqfck8+2Bi0uUAVYA51avec79WshJ3MbMqNO92wFzZFz0M0hy6Z1+54tfUpnhSsEigNy0HrYRSWZmRBGc5ydO240lgCO4IMhx4lCNQj24Tg6GOvTGhaKP9JQxv13xUWhNZTkXhnc+uzvVq8qDesTPpyZLksK4OSzQ5Kq5yagtaJ0glXyEw+9QBMcX9Xyg5BATM+96VTErH0BtskplPt2u34NfonK3zrpZ0SFZhCPfWpq0zAifMRZPGzmv5n5HJh9HSZ0W/CBT9FZ//SpVYuF9YF+ZlGZyd4HvbXe9Hz3sa7je7m1ny6K+QheUTWSERekE3yhuySPmHkC/lJfpHfwcfgc/A1+DazBq35mgdkqYLvfwAs/wN9</latexit> W = Z ⌦ z(r)⇤z(r)Tdr <latexit sha1_base64="ots8Wmdw3c6kEAqnCoYqjjAHV2E=">AAAD0HiclVJbb9MwFHYWLqNctsEjL9YqpgJb1UwT8II0ARIDhDbQuk6au8pxT1KrcRLZDmrJLMQjvPM7eIWfwr/BSdOKbhSJIyX+9J3v3HzspxFXutX65Sy5ly5fubp8rXb9xs1bK6trt49UkkkGbZZEiTz2qYKIx9DWXEdwnEqgwo+g4w+fF/7OB5CKJ/GhHqfQFTSMecAZ1ZbqrTnrr/EGoWkqkxEmPNa9nOwLCKnBJIJAn2Hii5wMrYKaRoGluX+aE0H1QAX5oal0jVL3xjwkI16gV5aXPBxoK97y5kQvTfHvm63iyEyVTFoKlJmFVefZ6Tae+c2kPiaktvG0SllmXJRs1sK0372igFV2zH+E91brrWarNHwReBWoo8oO7K1+I/2EZQJizSKq1InXSnU3p1JzFoGpkUxBStmQhnBiYUwFqG5ebtPge5bp4yCR9os1Ltk/I3IqlBoL3yrLqc77CvJvvpNMB0+6OY/TTEPMJoWCLMI6wcXTwH0ugelobAFlktteMRtQSZm2D6g2V8YXm+VFqUAZ63kBdkwJby21n4KkOpEPckJlKOjI2LFDslmgfwl5PBVatEhok3DBP4LJZ2ihlMdT6RTV7CK982u7CI62m96j5s67nfrus2qly+guWkcN5KHHaBftoQPURsz54nx3fjg/3ffuyP3kfp5Il5wq5g6aM/frbwuZR80=</latexit> J ⇡ Z ⌦ (r)T (K + ⇠I) 1 Gd udes 2 dr = Gd udes H W Gd udes <latexit sha1_base64="UqfxQe3UGsqqnmcvxH4VBmwhEN8=">AAAC/XicfZFdaxNBFIYn61eNH03rpTeDQYhaQ1aKFkEo6oUiYoSmLWRjODs5uxkyM7vMzErTZfBX+BO8Ey/1twje6u9wdpuIba0HBh7e952PMyfOBTe21/veCM6dv3Dx0srl5pWr166vttbWd01WaIYDlolM78dgUHCFA8utwP1cI8hY4F48e1b5e+9RG56pHTvPcSQhVTzhDKyXxq1+FMvy0NHHT2hF0QzyHFynYu3uvCsjCXZqknLHORoJTGynzr1y92h0wCt86Q3N06n16fuhG7favW6vLnoawgW0yaL647XGx2iSsUKiskyAMcOwl9tRCdpyJtA1o8JgDmwGKQ49KpBoRmXduqO3vTKhSab9UpbW6t87SpDGzGXsk3UnJ71K/Jc3LGyyNSq5yguLih1dlBSC2oxW/0gnXCOzYu4BmOb+rZRNQQOz/rebx66J5UZ1vjaJcd55jr5Nja+99CZHDTbTd8sIdCrhwPm202ijov8FuVoGPZ0V9IdwyQ/RlX/ozChXy+iSmn6Q4cmxnYbdB93wYXfz7WZ7++lipCvkJrlFOiQkj8g2eUH6ZEAY+Up+kJ/kV/Ah+BR8Dr4cRYPGYs8NcqyCb78Bcfj4Dw==</latexit> z := (r)T (K + ⇠I) 1 Kernel ridge regression <latexit sha1_base64="CRJvQ0OuNgbbSk0jTZw5dDlZAo8=">AAADxHiclVJbaxNBFJ7teqnx0ovgiy+DQanahqyU6otQrNgKihVMU8ikYXZyNhk6M7vMzIpxGf+Fv8BX/UH+G2e3G0nbRPDAwrff+c5tzokzwY1tt38HS+GVq9euL99o3Lx1+87K6tr6kUlzzaDDUpHq45gaEFxBx3Ir4DjTQGUsoBuf7pX+7mfQhqfqk51k0Jd0pHjCGbWeGqwF98iY2oLEshg6hx+9xITqkeRqUHOEK0wkteM4LvbcyTuHiYDEbuDSvT90eKtCuTspKpn2QWB8KqL5aGwf17RJioOS9NLu/6bAhDTKxmaC3Jy0+w4/xQQsLf/ezjSwFbnFUW5+7cFqs91qV4Yvg6gGTVTboX/J72SYslyCskxQY3pRO7P9gmrLmQDXILmBjLJTOoKeh4pKMP2i2qDDDz0zxEmq/acsrtjZiIJKYyYy9spqgIu+kpzn6+U2edEvuMpyC4qdFUpygW2Ky3PAQ66BWTHxgDLNfa+YjammzPqjaZwrE8vN6oVMYpz3vAY/pob3nvqQgaY21U+K6njoF+fHHpHNEv1LyNVU6NEioU/CJf8KrviLFkq5mkqnqOEXGV1c22Vw9KwV7bS2P243d1/VK11G99EDtIEi9BztogN0iDqIBS74EfwMfoVvQhGaMD+TLgV1zF10zsJvfwD8G0Dj</latexit> ˆ d = arg min d2CL Gd udes H W Gd udes = GHW G + ⌘I 1 GHW udes [Koyama+ JAES 2023]

Weighted Pressure Matching Ø Comparison between Pressure Matching and Weighted
Pressure Matching September 3, 2024 36 PM WPM (uniform) WPM (directional) Pressure Error [Koyama+ JAES 2023]

Spatial aliasing artifacts in sound field control Ø Owing to
discrete placement of secondary sources, spatial aliasing artifacts are unavoidable – E.g., Synthesizing sound field by 12 loudspeakers at 800 Hz September 3, 2024 37 Desired Pressure Matching Pressure § Degradation of sound localization § Coloration of source signals

Idea for perceptual quality enhancement Ø Interaural level difference (ILD)
is the dominant cue for horizontal sound localization above 1500 Hz, compared with interaural time difference (ITD) Ø Amplitude response should be accurately synthesized as much as possible, rather than phase response, to alleviate coloration effects September 3, 2024 38 Synthesizing amplitude (or magnitude) distribution leaving phase distribution arbitrary at high frequencies Applying amplitude matching for high frequencies Pressure Magnitude

Amplitude matching Ø Synthesizing desired amplitude at control points –
By leaving phase arbitrary, number of parameters to be control can be reduced – First proposed for multizone sound field control for personal audio Ø Optimization problem of amplitude matching September 3, 2024 39 : Target region Secondary source [Koyama+ 2021, Abe+ 2023] Desired amplitude No closed-form solution, but majorization minimization (MM) algorithm or alternating direction method of multipliers (ADMM) can be applied Element-wise absolute value

Amplitude matching September 3, 2024 40 https://youtu.be/oYw7kmpZcY4 Full version:

Proposed method for perceptual quality enhancement Ø Combination of pressure
and amplitude matching – is determined so that for low frequencies and for high frequencies – For example, can be deﬁned as sigmoid function September 3, 2024 41 <latexit sha1_base64="7PBZJNy5EyF+eucvkOfjzch+WsM=">AAADWHicfZLtbtMwFIadFthWPtaOn/yxqJA6WKummhiaNKkwJBACMSS6TarbynFOO2uxE9kOoqS+DK6CK4KrwUlTYF9YivLqPc/x8fFxkERcm273p1ep3rp9Z219o3b33v0Hm/XG1rGOU8VgwOIoVqcB1RBxCQPDTQSniQIqgghOgvPDPH7yBZTmsfxs5gmMBJ1JPuWMGmdN6j+I4JIL/g0mGQlEFlrCJRHUnAVBdmjH7y1+11oGtvH+AW75bRKAoduYLHL7TWjb+T+146xIUw4FbS1ZTHrjHn6GC9zRK3yB23hxbcrib07kOghpWSMsN5vUm91Ot1j4qvBL0UTlOpo0vO8kjFkqQBoWUa2Hfjcxo4wqw1kEtkZSDQll53QGQyclFaBHWXGrFj9xToinsXKfNLhw/83IqNB6LgJH5l3oy7HcvC42TM30xSjjMkkNSLYsNE0jbGKcjwiHXAEz0dwJyhR3Z8XsjCrKjBtk7UKZQOwUN6in2rrIa3BtKvjgrI8JKGpi9TQjVM0E/Wpd2zOyk6v/gVyuQKduAt0mxZux2R91I1o+L4euVM0N0r88tqviuNfxn3d2P+02+6/Kka6jR+gxaiEf7aE+eouO0AAxr+HteX3vZeVXFVXXqhtLtOKVOQ/RhVXd+g1mbRVc</latexit> minimize d2CL J(d) := (1 )kGd udesk2 2 + k|Gd| |udes|k2 2 + kdk2 2 <latexit sha1_base64="A4dv1PYDkl/3NJ52frUgh4Qg2ME=">AAACsHicfZFNSyNBEIY7o66a1fVjj14Gg7CIhBkR9SjuHvayqGA0kAmhplMTW7t7hu4aMTvMb9iToD/Nf7M9MRG/Cxoe3nqru6orzqSwFAQPNW9qeubL7Nx8/evC4rel5ZXVM5vmhmOLpzI17RgsSqGxRYIktjODoGKJ5/HVzyp/fo3GilSf0jDDroKBFongQE5qRTES9JYbQTMYhf8WwjE02DiOeyu126if8lyhJi7B2k4YZNQtwJDgEst6lFvMgF/BADsONSi03WLUbelvOKXvJ6lxR5M/Up9XFKCsHarYORXQhX2dq8T3cp2ckv1uIXSWE2r++FCSS59Svxrd7wuDnOTQAXAjXK8+vwADnNwH1V88E6ut6n5jE1u6zC90Yxr846SjDA1QajaLCMxAwU3pxh5EWxV9ZhR6YnT0kdFdIpT4i2XxRB9ahZ5YJ1R3iwxfr+0tnG03w93mzslO4+BwvNI5tsbW2Q8Wsj12wH6zY9ZinAn2j92xe2/ba3s9Dx6tXm1c8529CO/yPyvr2Vk=</latexit> <latexit sha1_base64="v6ZpLa9I3hnLLs2O89d3S1l6XjM=">AAACsnicfZFNaxsxEIblzUdTN9899rLEFEIIZjeYNJdAaHPopTSBOA7xmjArzzrCknaRZkPcZX9ED7m0vyz/JlrHDvkeEDy88440o4kzKSwFwW3Nm5mdm/+w8LH+aXFpeWV1bf3Uprnh2OapTM1ZDBal0NgmQRLPMoOgYomdePijyneu0FiR6hMaZdhTMNAiERzISZ0oRoL98GK1ETSDcfgvIZxAg03i6GKtdhP1U54r1MQlWNsNg4x6BRgSXGJZj3KLGfAhDLDrUINC2yvG/Zb+V6f0/SQ17mjyx+rjigKUtSMVO6cCurTPc5X4Wq6bU7LXK4TOckLN7x9KculT6lfD+31hkJMcOQBuhOvV55dggJP7ovqTZ2K1Xd1vbGJLlzlEN6bBX076naEBSs1WEYEZKLgu3diDaLui94xCT42O3jK6S4QSf7AsHuhNq9BT65TqbpHh87W9hNOdZrjbbB23GgffJytdYF/YBttkIfvGDthPdsTajLMh+8v+sf9eyzv3wOP3Vq82qfnMnoQn7wB2ednb</latexit> = 1 <latexit sha1_base64="nFwv8TaL4YCj47fcATfYtfC3o0Y=">AAACsnicfZFNSyNBEIY74+6qcT/8OHoZNgjLImFGwrqXBVEPXkQFY2QzQWo6NbFJd8/QXbNsHOZHePCiv8x/Y09MxO+Choe33uqu6oozKSwFwW3Nm/nw8dPs3Hx94fOXr98Wl5ZPbJobjm2eytScxmBRCo1tEiTxNDMIKpbYiYc7Vb7zD40VqT6mUYY9BQMtEsGBnNSJYiT4E5wtNoJmMA7/JYQTaLBJHJ4t1a6ifspzhZq4BGu7YZBRrwBDgkss61FuMQM+hAF2HWpQaHvFuN/SX3NK309S444mf6w+rihAWTtSsXMqoHP7PFeJr+W6OSW/e4XQWU6o+f1DSS59Sv1qeL8vDHKSIwfAjXC9+vwcDHByX1R/8kys1qv7jU1s6TK76MY0uO+kgwwNUGp+FhGYgYL/pRt7EK1X9J5R6KnR0VtGd4lQ4gLL4oHetAo9tU6p7hYZPl/bSzjZaIa/mq2jVmNre7LSObbKvrMfLGSbbIvtsUPWZpwN2SW7Zjdey/vrgcfvrV5tUrPCnoQn7wB0Mdna</latexit> = 0 <latexit sha1_base64="A4dv1PYDkl/3NJ52frUgh4Qg2ME=">AAACsHicfZFNSyNBEIY7o66a1fVjj14Gg7CIhBkR9SjuHvayqGA0kAmhplMTW7t7hu4aMTvMb9iToD/Nf7M9MRG/Cxoe3nqru6orzqSwFAQPNW9qeubL7Nx8/evC4rel5ZXVM5vmhmOLpzI17RgsSqGxRYIktjODoGKJ5/HVzyp/fo3GilSf0jDDroKBFongQE5qRTES9JYbQTMYhf8WwjE02DiOeyu126if8lyhJi7B2k4YZNQtwJDgEst6lFvMgF/BADsONSi03WLUbelvOKXvJ6lxR5M/Up9XFKCsHarYORXQhX2dq8T3cp2ckv1uIXSWE2r++FCSS59Svxrd7wuDnOTQAXAjXK8+vwADnNwH1V88E6ut6n5jE1u6zC90Yxr846SjDA1QajaLCMxAwU3pxh5EWxV9ZhR6YnT0kdFdIpT4i2XxRB9ahZ5YJ1R3iwxfr+0tnG03w93mzslO4+BwvNI5tsbW2Q8Wsj12wH6zY9ZinAn2j92xe2/ba3s9Dx6tXm1c8529CO/yPyvr2Vk=</latexit> <latexit sha1_base64="ROtRu2ks9BH5ktUl1T0BgfcB3qQ=">AAADB3icfVHNbhMxEHaWv7L8peXIxSJCaqGNdqsKekGqgAMXRJGatiIOkdeZ3Vpd2yvbiwiWH4AH4Dm4IY7wGLwAV3gEvMkG0ZYykuVP33zjGX+TVSU3Nkm+d6ILFy9dvrJ0Nb52/cbNW93llX2jas1gwFSp9GFGDZRcwsByW8JhpYGKrISD7Phpkz94C9pwJffstIKRoIXkOWfUBmrcfU0ysHSVKAEFXcOPcUxyTZlLvUsfEEHtkRYO/Bu3MeeJ4YWg3m2Sivu2bGN+jd1Cv+f9mvfjbi/pJ7PAZ0Hagh5qY3e83PlIJorVAqRlJTVmmCaVHTmqLWcl+JjUBirKjmkBwwAlFWBGbmaCx/cCM8G50uFIi2fs3xWOCmOmIgvKZkxzOteQ/8oNa5tvjxyXVW1BsnmjvC6xVbhxFE+4BmbLaQCUaR5mxeyIBrNs8D0+0SYT6zOLTG58yDyD8E0NLwL1sgJNrdL3HaE6OPzOh28XZL1B/xNyuRAGdJ4wPMIFfw/e/UHnSrlcSBcoDotMT6/tLNjf7KcP+1uvtno7T9qVLqE76C5aRSl6hHbQc7SLBoihr+gH+ol+RR+iT9Hn6MtcGnXamtvoRETffgMYmvzf</latexit> (!) = 1 1 + e 2⇡ (! !T) Transition frequency Can still be solved by MM algorithm or ADMM Pressure matching Amplitude matching

Numerical experiments Ø Setting – 3D free ﬁeld – Target
region : Cuboid of 1.0 m x 1.0 m x 0.04 m – 32 loudspeakers on borders of squares of 2.0 m x 2.0 m at z=±0.1 m – 1152 control points regularly placed over every 0.04 m – Desired sound ﬁeld: point source at (2.0 m, 0.0 m, 0.0 m) – Proposed method and pressure matching (PM) are compared September 3, 2024 42 <latexit sha1_base64="aJVm6ibCtbqRlu7KgruECMVr7Qw=">AAACvHicfZFNT+MwEIbdwAJbdvk87iWiQkJoVSUIASeEgAMXVFaiBamp0MSdBFPbiWwHUaL8B65w4m/xb3D6saJ8jWTp0TuvPTOeMOVMG897qThT0z9mZud+Vud//V5YXFpeaekkUxSbNOGJugxBI2cSm4YZjpepQhAhx4uwd1TmL25RaZbIc9NPsSMglixiFIyVWkFDYAxXSzWv7g3C/Qj+CGpkFGdXy5XnoJvQTKA0lIPWbd9LTScHZRjlWFSDTGMKtAcxti1KEKg7+aDdwl23SteNEmWPNO5AfXsjB6F1X4TWKcBc6/e5Uvws185MtNfJmUwzg5IOC0UZd03ilrO7XaaQGt63AFQx26tLr0EBNfaHJqqEYmKGvKyldKSLajU4RjuywlMrNVJUYBK1mQegYgF3hf2COPhb0ndGJsdGS18Z7SNMsHss8v/0pZXJsXVMdqf++w1+hNZW3d+pb//brh0cjrY7R/6QNbJBfLJLDsgJOSNNQskNeSCP5MnZd7pOzxFDq1MZ3VklE+HcvgLkeN8p</latexit> ⌦ <latexit sha1_base64="aJVm6ibCtbqRlu7KgruECMVr7Qw=">AAACvHicfZFNT+MwEIbdwAJbdvk87iWiQkJoVSUIASeEgAMXVFaiBamp0MSdBFPbiWwHUaL8B65w4m/xb3D6saJ8jWTp0TuvPTOeMOVMG897qThT0z9mZud+Vud//V5YXFpeaekkUxSbNOGJugxBI2cSm4YZjpepQhAhx4uwd1TmL25RaZbIc9NPsSMglixiFIyVWkFDYAxXSzWv7g3C/Qj+CGpkFGdXy5XnoJvQTKA0lIPWbd9LTScHZRjlWFSDTGMKtAcxti1KEKg7+aDdwl23SteNEmWPNO5AfXsjB6F1X4TWKcBc6/e5Uvws185MtNfJmUwzg5IOC0UZd03ilrO7XaaQGt63AFQx26tLr0EBNfaHJqqEYmKGvKyldKSLajU4RjuywlMrNVJUYBK1mQegYgF3hf2COPhb0ndGJsdGS18Z7SNMsHss8v/0pZXJsXVMdqf++w1+hNZW3d+pb//brh0cjrY7R/6QNbJBfLJLDsgJOSNNQskNeSCP5MnZd7pOzxFDq1MZ3VklE+HcvgLkeN8p</latexit> ⌦

Numerical experiments Ø Evaluation of ILD – Binaural signals in
the synthesized sound ﬁeld were calculated by using transfer functions from loudspeakers to a listener obtained by Mesh2HRTF [Ziegelwanger+ 2015] – Evaluation measure was normalized error of ILD: – Distribution of NE September 3, 2024 43 <latexit sha1_base64="EndMQejlYdBkvqIegxmgN2TX4IM=">AAAD5niclVLdatRAFJ40amv8a+ulN4OL0Eq7JFLa3hSKVqjgTwW3LWyWZTJ7sjs0MwkzE3Gdzit4pXjb5/BWX8K3cbLdSLe7FXsg8OU73/mZc05SZEzpMPztzfk3bt6aX7gd3Ll77/6DxaXlQ5WXkkKL5lkujxOiIGMCWprpDI4LCYQnGRwlJy8q/9FHkIrl4oMeFtDhpC9YyijRjuoueesxJ3oguXn70q7ECTfSdk3N7Vu7indwEKeSUBOrkjtfMWATCotP679Xr/cuRKuhsDNzrk3lWF2fnULLEv47xym21+7xegVsd7ERNsOR4WkQjUEDje3ADfgs7uW05CA0zYhS7SgsdMcQqRnNwAZxqaAg9IT0oe2gIBxUx4wWa/ETx/Rwmkv3CY1H7MUIQ7hSQ544ZdWmuuyryFm+dqnT7Y5hoig1CHpeKC0zrHNcXQnuMQlUZ0MHCJXM9YrpgLgj0O6WJqokfOINZjQulSobBPEeuCdLeOOodwVIonP51MRE9jn5ZN0I+vFahf4lZKIWOnSV0CVhnH0Ga/6iK6VM1NIauZ1Glzc4DQ6fNaPN5sb7jcbu8/F2F9Aj9BitoAhtoV20jw5QC1Hvq/fD++n98gf+F/+b//1cOueNYx6iCfPP/gCusFhL</latexit> NE(r H ) = P H |ILD syn (r H , H ) ILD true (r H , H )| P H |ILD true (r H , H )| Position and direction of listener’s head PM Proposed

Numerical experiments Ø Evaluation of amplitude response – At origin
September 3, 2024 44

Listening experiments Ø Evaluation by MUSHRA – Desired sound field:
point source at (2.0 m, 0.5 m, 0.0 m) – Reverberation time (T60 ): 0.19 s – 14 male subjects in 20-30s – Listening at center of target region – Test signals: • Reference: Source signal from reference loudspeaker • C1/Hidden anchor: lowpass-filtered source signal up to 3.5 kHz • C2/PM: Synthesized sound by PM • C3/Proposed: Synthesized sound by Proposed • C4/Hidden reference: Same as reference September 3, 2024 45

Listening experiments Ø Results of two source signals (Vocals/Instrumental) September
3, 2024 46 Vocals Instrumental C1/Hidden anchor C2/PM C3/Proposed C4/Hidden reference Synthesized sound by Proposed is perceptually close to reference sound compared to PM

Application to Spatial Active Noise Control Ø Environmental noise is
still unsolved problem Ø Active noise control (ANC) is aimed to cancel noise by loudspeaker signals, but its effect is limited to local region Ø ANC in 3D space based on sound field analysis/synthesis September 3, 2024 47 Noise suppression by loudspeaker signals Quiet zone

Application to Spatial Active Noise Control Ø Cost function of
regional noise power is estimated by kernel interpolation of sound field Ø Adaptive filtering algorithm based on kernel interpolation is also derived September 3, 2024 48 ANC in 3D space based on sound ﬁeld interpolation Ø Conventional cost function Ø Proposed cost function <latexit sha1_base64="Pnlxe5THpW3gUCE+pMJvaUIbsmQ=">AAACFXicbZDLSsNAFIYnXmu9RV2KMFgUVyUpRd0IRTcuXFSwF2himUwn7dCZJMxMhBKz8iF8Bre6diduXbv0TZy0WdjWHwY+/nMO58zvRYxKZVnfxsLi0vLKamGtuL6xubVt7uw2ZRgLTBo4ZKFoe0gSRgPSUFQx0o4EQdxjpOUNr7J664EIScPgTo0i4nLUD6hPMVLa6poHDkdqgBFLblJ4fAGdR+h4PCGppvtK1yxZZWssOA92DiWQq941f5xeiGNOAoUZkrJjW5FyEyQUxYykRSeWJEJ4iPqkozFAnEg3GX8jhUfa6UE/FPoFCo7dvxMJ4lKOuKc7s6PlbC0z/6t1YuWfuwkNoliRAE8W+TGDKoRZJrBHBcGKjTQgLKi+FeIBEggrndzUFo+nOhN7NoF5aFbK9mm5elst1S7zdApgHxyCE2CDM1AD16AOGgCDJ/ACXsGb8Wy8Gx/G56R1wchn9sCUjK9f5U+ekQ==</latexit> L = kek2 : Power of error mics : Regional noise power <latexit sha1_base64="pm/hhsXKsYUZ/lWoZalEg8K5t2U=">AAACNHicbVDLSgMxFM34rPVVdekmWJS6KTOlqAuFohsXghXsAzq1ZNK0DU0yQ5IRynR+xo/wG9zqTnAj4tZvMNN2YVsPBA7nnMu9OV7AqNK2/W4tLC4tr6ym1tLrG5tb25md3aryQ4lJBfvMl3UPKcKoIBVNNSP1QBLEPUZqXv8q8WuPRCrqi3s9CEiTo66gHYqRNlIrc+5ypHsYsegmhkcX0KVCtyL3lpMuiuEwzLkej2R8PHwowFFU8qgdj0XYymTtvD0CnCfOhGTBBOVW5tNt+zjkRGjMkFINxw50M0JSU8xInHZDRQKE+6hLGoYKxIlqRqNfxvDQKG3Y8aV5QsOR+nciQlypAfdMMjlUzXqJ+J/XCHXnrBlREYSaCDxe1AkZ1D5MKoNtKgnWbGAIwpKaWyHuIYmwNsVObfF4bDpxZhuYJ9VC3jnJF++K2dLlpJ0U2AcHIAcccApK4BqUQQVg8ARewCt4s56tD+vL+h5HF6zJzB6YgvXzC9DNrBs=</latexit> L = Z ⌦ |u(r)|2dr [Ito+ IEEE ICASSP 2019 (Best Student Paper Award), Koyama+ IEEE/ACM TASLP 2021]

Application to Spatial Active Noise Control Ø Band-limited noise (500-800Hz),
T60 : 240ms September 3, 2024 49 Proposed: -10.5 dB MPC: -6.2 dB (dB) Regional noise reduction is achieved by the proposed method

Conclusion Ø Physics-informed sound field estimation and control qSound field
estimation: • Kernel interpolation of sound field with constraint of Helmholtz equation • Kernel function adapted to acoustic environment with the aid of neural networks qSound field control: • Combination of pressure and amplitude matching for perceptual quality enhancement • Spatial active noise control based on kernel interpolation September 3, 2024 50 Thank you for your attention!

Physics-Informed Machine Learning For Sound Fie...

Physics-Informed Machine Learning For Sound Field Estimation and Control

More Decks by NII S. Koyama's Lab

Other Decks in Research

Featured

Transcript