Physics-Informed Machine Learning For Sound Field Estimation and Control

Seminar Talk at Polytechnic University of Milan

Transcript

1. Physics-Informed Machine Learning For Sound Field Estimation and Control Shoichi Koyama National Institute of Informatics, Tokyo, Japan
2. About me Ø Shoichi Koyama, Ph.D. • 2009 ‒ 2014: Researcher, NTT Labs • 2014 ‒ 2018: Research Associate, UTokyo • 2016 ‒ 2018: Visiting Researcher, Paris Diderot Univ. • 2018 ‒ 2023: Lecturer, UTokyo • 2020 ‒ 2023: Visiting Associate Prof., Tohoku Univ. • 2023 ‒ current: Associate Prof., NII September 3, 2024 2
3. About NII Ø NII is national research institute of informatics in Japan – Main lab is located in central Tokyo – Associated with graduate university called SOKENDAI September 3, 2024 3 Kashiwa Annex NII 29 km Join us as an intern or PhD student!
4. September 3, 2024 4 Basic Technologies of Sound Field Estimation and Control VR/AR audio Active noise control Local-field recording and reproduction Signal enhancement Visualization/auralization Room acoustic analysis Sound field estimation/control and its applications
5. What is sound field estimation/control? September 3, 2024 5 Estimating sound field inside target region using multiple mics Synthesizing desired sound field inside target region using multiple loudspeakers Physics-informed signal processing/machine learning for sound field estimation and control Estimation Control Microphone Loudspeaker
6. SOUND FIELD ESTIMATION September 3, 2024 6
7. Sound field estimation September 3, 2024 7 Formulation of sound field estimation problem Estimate pressure distribution with observations at discrete set of mics in the frequency domain : Source-free and simply-connected interior region Microphone Target region:
8. Sound field estimation Ø General interpolation techniques – Representing by model parameters as – Solve the optimization problem September 3, 2024 8 Formulation of sound field estimation problem Microphone Target region: <latexit sha1_base64="QB9up2Xq4KtGilt/H+nRpzcDgKk=">AAADJnicfVLdahQxFM6MP63rT3f10pvgImx1WWZEqiBCUS+8sFjFbQubdchkz8yGJjNDkhHXkKfwJXwa70S80zcxM93qtrUeCHx85zv/SSvBtYmiH0F44eKly2vrVzpXr12/sdHt3dzTZa0YjFkpSnWQUg2CFzA23Ag4qBRQmQrYTw+fN/79D6A0L4t3ZlHBVNK84Bln1Hgq6VoiecEl/wSJJam0xMzBUOcwkdTMGRX2lSMCMjPAxOJ60GiUS+QTvKLexMQlVj6N3fudYevQPoHi+dxs4vt/U711g9WopNuPRlFr+CyIl6CPlrab9ILPZFayWkJhmKBaT+KoMlNLleFMgOuQWkNF2SHNYeJhQSXoqW235PBdz8xwVir/CoNbdjXCUqn1QqZe2TSsT/sa8l++SW2yx1PLi6o2ULCjQlktsClxs3I84wqYEQsPKFPc94rZnCrKjD9M50SZVA6b/Epn2k/zAvyUCnY887oCRU2p7llCVS7pR+enzsmwQf8T8uJY6NF5Qp+k/QLO/kHnSpe/xUuPkT9jfPpoZ8Heg1G8Ndp687C//Wx50HV0G91BAxSjR2gbvUS7aIwY+hWsBd2gF34Jv4bfwu9H0jBYxtxCJyz8+RuXLAcP</latexit> minimize ✓ L {u(rm; ✓)}M m=1 , s + R(✓) <latexit sha1_base64="bgfM3WFHH+06JgGQdjycQWMo7ww=">AAACwXicfZHBbhMxEIadhUIILU1A6oWLRVSprapoF6GCxKWiPfSCaKWmrZSNollnNrFqe1f2LCIs+xS8AVd4Id6m3jRBtKWMZPnTP//YHk+SK+koDH83ggcPVx49bj5pPV1de7be7jw/c1lhBfZFpjJ7kYBDJQ32SZLCi9wi6ETheXJ5UOfPP6N1MjOnNMtxqGFiZCoFkJdG7Y1iK050aav3vN5jmiJBtT1qd8NeOA9+F6IFdNkijkedxvd4nIlCoyGhwLlBFOY0LMGSFAqrVlw4zEFcwgQHHg1odMNy3kDFN70y5mlm/TLE5+rfFSVo52Y68U4NNHW3c7X4r9ygoPTdsJQmLwiNuL4oLRSnjNe/wcfSoiA18wDCSv9WLqZgQZD/s9aNaxK9W59vXep8N4fou7T40SufcrRAmd0pY7ATDV8q3/Uk3q3pf0ZplkZP9xn9IVLLr1iVf+heqzRL65L8GKPbQ7sLZ6970V5v7+RNd//DYqBN9pK9YlssYm/ZPjtix6zPBPvGfrCf7FdwEMggD+y1NWgsal6wGxGUV4wM4Gk=</latexit> u(r; ✓) <latexit sha1_base64="yLksvcVXn4jdtyNYM7Soz6yW9Ko=">AAACqnicfZHbSgMxEIbT9VxPrV56s1gEkVJ2RaqXRb3wRjxgq9IWmU1nazDJLklWrEufwEv14Xwbs7UVrYeBwMc//yQzmSDmTBvPe8s5E5NT0zOzc/n5hcWl5UJxpaGjRFGs04hH6ioAjZxJrBtmOF7FCkEEHC+Du4Msf3mPSrNIXphejG0BXclCRsFY6Sy5KZS8ijcI9yf4QyiRYZzeFHPPrU5EE4HSUA5aN30vNu0UlGGUYz/fSjTGQO+gi02LEgTqdjrotO9uWKXjhpGyRxp3oH6tSEFo3ROBdQowt3o8l4m/5ZqJCffaKZNxYlDSj4fChLsmcrOx3Q5TSA3vWQCqmO3VpbeggBr7OflvzwSinN2vdKjtNIdop1R4bJWTGBWYSG2lLVBdAQ99O3W3Vc7oPyOTI6Olv4z2EibYI/bTT/rTyuTIOiK7Rn98aT+hsV3xq5Xq2U6ptj9c6CxZI+tkk/hkl9TIETkldUIJkifyQl6dsnPuXDvND6uTG9askm/hdN4Bu6fXkA==</latexit> u <latexit sha1_base64="ru0OGkVwXcC9n3iBv3AvtKvDAnk=">AAACtHicfZFdSxtBFIYnW1vTaFttL71ZDAURCbtSopeh7YU3RQtGhWyQs5OzyeB8LDNnxbjsryi9rb+r/6azMRE/e2Dg4T3vzJyPNJfCURT9bQSvll6/WW6+ba2svnv/YW3944kzheXY50Yae5aCQyk09kmQxLPcIqhU4ml68a3On16idcLoY5rmOFQw1iITHMhLgyRVZUITJKjO19pRJ5pF+BTiObTZPI7O1xu/kpHhhUJNXIJzgzjKaViCJcElVq2kcJgDv4AxDjxqUOiG5azmKvzslVGYGeuPpnCm3r9RgnJuqlLvVEAT9zhXi8/lBgVl+8NS6Lwg1Pz2o6yQIZmwHkA4EhY5yakH4Fb4WkM+AQuc/JhaD75J1U79vnWZ8918R9+lxR9eOczRAhm7XSZgxwquKt/1ONmp6X9GoRdGTy8Z/SNCiWusyjt60Sr0wrogv8b48dKewsluJ+52uj+/tHtf5wttsg22ybZYzPZYjx2wI9ZnnBn2m/1hN0E3SAIe4K01aMzvfGIPItD/AKw13BQ=</latexit> ✓ Loss function for observation Regularization term for <latexit sha1_base64="ru0OGkVwXcC9n3iBv3AvtKvDAnk=">AAACtHicfZFdSxtBFIYnW1vTaFttL71ZDAURCbtSopeh7YU3RQtGhWyQs5OzyeB8LDNnxbjsryi9rb+r/6azMRE/e2Dg4T3vzJyPNJfCURT9bQSvll6/WW6+ba2svnv/YW3944kzheXY50Yae5aCQyk09kmQxLPcIqhU4ml68a3On16idcLoY5rmOFQw1iITHMhLgyRVZUITJKjO19pRJ5pF+BTiObTZPI7O1xu/kpHhhUJNXIJzgzjKaViCJcElVq2kcJgDv4AxDjxqUOiG5azmKvzslVGYGeuPpnCm3r9RgnJuqlLvVEAT9zhXi8/lBgVl+8NS6Lwg1Pz2o6yQIZmwHkA4EhY5yakH4Fb4WkM+AQuc/JhaD75J1U79vnWZ8918R9+lxR9eOczRAhm7XSZgxwquKt/1ONmp6X9GoRdGTy8Z/SNCiWusyjt60Sr0wrogv8b48dKewsluJ+52uj+/tHtf5wttsg22ybZYzPZYjx2wI9ZnnBn2m/1hN0E3SAIe4K01aMzvfGIPItD/AKw13BQ=</latexit> ✓ <latexit sha1_base64="wnDK9ytRrHjhZwyUfqPJAj2Ykik=">AAAC03icfZHLahRBFIZrOlHjeJvo0k3hIAQZhm6R6EYIxoWbYIRMEphum9M1pydF6tJUVUsmRW/ErW58Fbf6Hr6N1ZOZYBLjgYKP//xVdS5FJbh1cfy7E62s3rh5a+12987de/cf9NYf7ltdG4YjpoU2hwVYFFzhyHEn8LAyCLIQeFAcb7f5g09oLNdqz80qzCRMFS85AxekvLeRFtLbhr6mY5snA5qKiXZ2QG2+k330qQR3ZEu/1zR5rx8P43nQq5AsoE8WsZuvd76nE81qicoxAdaOk7hymQfjOBPYdNPaYgXsGKY4DqhAos38vKWGPg3KhJbahKMcnat/3/AgrZ3JIjjnNV7OteK/cuPala8yz1VVO1Ts7KOyFtRp2s6HTrhB5sQsADDDQ62UHYEB5sIUuxe+KeSgfd/Y0oZu3mLo0uBOUN5XaMBp88ynYKYSTprQ9TQdtPQ/I1dLY6DrjOERLvkpNv6crrVytbQuKawxuby0q7D/fJhsDjc/vOhvvVksdI08Jk/IBknIS7JF3pFdMiKMfCM/yE/yKxpFPvocfTmzRp3FnUfkQkRf/wAenuda</latexit> s = [s1, . . . , sM ]T
9. Sound field estimation Ø Function to be interpolated should satisfy Helmholtz eq – Homogeneous Helmholtz equation in source-free target region – Conventional approach: expansion into element solutions of Helmholtz eq • Plane wave expantion (or Herglotz wave function) • Spherical wave function expansion • Equivalent source distribution (or single layer potential) September 3, 2024 9 What kind of physical properties can be embedded? <latexit sha1_base64="3MQnLgSPT9UXBsXZ4NqNl8viJ5k=">AAACynicfZHPThsxEMadhbY0/Rfg2EOtRpVoi6LdCAEXJEQ59FBUKjWAlA3RrDMbrNjele1FBGtv9Al4DK70Yfo29YakKlA6kqWfvvnG9swkueDGhuGvWjA3/+jxk4Wn9WfPX7x81VhcOjBZoRl2WCYyfZSAQcEVdiy3Ao9yjSATgYfJ6FOVPzxFbXimvttxjj0JQ8VTzsB6qd94sxIrSAT0XZxIp8vyuE0/0tFx+z0t6BYN+41m2AonQe9DNIUmmcZ+f7F2GQ8yVkhUlgkwphuFue050JYzgWU9LgzmwEYwxK5HBRJNz006Kek7rwxomml/lKUT9e8KB9KYsUy8U4I9MXdzlfivXLew6WbPcZUXFhW7eSgtBLUZrcZCB1wjs2LsAZjm/q+UnYAGZv3w6reeSeRqdb82qfHd7KLvUuOeV77mqMFm+oOLQQ8lnJW+62G8WtH/jFzNjJ4eMvpLuOTnWLo/9KCVq5l1Rn6N0d2l3YeDditab61/W2tu70wXukBek7dkhURkg2yTz2SfdAgjP8gVuSY/gy+BDsaBu7EGtWnNMrkVwcVvo0niRw==</latexit> (r2 r + k2)u = 0 Linear combination of element solutions is still a solution of Helmholtz eq
10. Basis expansion for sound field estimation Ø Plane wave expansion (or Herglotz wave function) September 3, 2024 10 <latexit sha1_base64="X5z1xMoVBReL3dH3nbvjSRQTg+g=">AAADMXicfZHNbhMxEMe9y1cJX2k5loNFhFRQiLIVKlyQKuDABVEEaSvFIfI6s1sT27uyvYhg+coL8Cg8TW+IKzwE3s0HtKWMZOnvmd94PDNpKbix/f5xFF+4eOnylbWrrWvXb9y81V7f2DdFpRkMWCEKfZhSA4IrGFhuBRyWGqhMBRyk0+d1/OAjaMML9c7OShhJmiuecUZtcI3bX6otkkqn/X38FBOu7NgRSe1Rmrq3frztMbFcTMBVvuEIWBrQBtHSgX/vHi4vHzyeYiKoygXgFdzF8/cx0U3Er5In/g81bnf6vX5j+KxIFqKDFrY3Xo++kknBKgnKMkGNGSb90o4c1ZazUKVFKgMlZVOawzBIRSWYkWsG5vG94JngrNDhKIsb798ZjkpjZjINZP1ZczpWO/8VG1Y2ezJyXJWVBcXmhbJKYFvgevp4wjUwK2ZBUKZ5+CtmR1RTZsOOWifKpLLbDMpkJnTzAkKXGl4Fz+sSNLWFfuAI1bmkn3zoOifdWv0P5GoJBnUeGB7hkn8G71bqXJSrJbpUYY3J6aWdFfvbvWSnt/PmUWf32WKha2gT3UVbKEGP0S56ifbQADH0K9qINqM78bf4OP4e/5ijcbTIuY1OWPzzN9y1DKA=</latexit> u(r) = Z S 2 ˜ u(⌘)e jkh⌘,rid⌘ -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -1 -0.5 0 0.5 1 Plane wave arrival direction
11. Basis expansion for sound field estimation Ø Spherical wave function expansion September 3, 2024 11 <latexit sha1_base64="GDd9eLOC2AN8O5PiWLeacMEt4Ac=">AAADinicfVFdb9MwFHUWPkYZ0MEjLxYVUou6KZ3QAKFKEwyJF8SQ6DZUl8pxndQ0diJ/IIqXX8Gv44m/wk3aIvbRWUp0dM655/r6xkUmjI2i38FGeOPmrdubdxp3t+7df9DcfnhscqcZH7A8y/VpTA3PhOIDK2zGTwvNqYwzfhLP3lb6yXeujcjVZzsv+EjSVIlEMGqBGjf/uDaJpddlB/cxMU6OPVGuH5VfPREqsfNyxUrX3wGpEuCPiaR2qoVKvSuXEZWpIqXPS8j7VkeV7Rk5W+g7V9jIWQd/qY1d6ACxGU9sG5NEU+bXl5X++lAI0iKd2s642Yp2o/rgy6C3BC20PEfj7eAXmeTMSa4sy6gxw15U2JGn2gqW8bJBnOEFZTOa8iFARSU3I19vosRPgZngJNfwKYtr9v8KT6UxcxmDs7quuahV5FXa0Nnk5cgLVTjLFVs0SlyGbY6rteKJ0JzZbA6AMi3grphNKTyiheU3zrWJZbd+KpMYmOaQw5SafwDmY8E1tbl+5gnVqaQ/Spg6Jd0KXWcUamUEtM4IIUKKn7z0/9Baq1Ar6wrBGnsXl3YZHO/t9vZ39z89bx28WS50Ez1GT1Ab9dALdIDeoyM0QCx4F8wCG7hwK9wLX4WvF9aNYFnzCJ074eFfxqEvpQ==</latexit> u(r) = 1 X ⌫=0 ⌫ X µ= ⌫ ˚ u(ro)j⌫(kkr ro k)Y⌫,µ ✓ r ro kr ro k ◆ -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 -0.4 -0.2 0 0.2 0.4 x (m) -0.4 -0.2 0 0.2 0.4 y (m) -0.1 -0.05 0 0.05 0.1 Expansion center Spherical Bessel function Spherical harmonic function
12. Basis expansion for sound field estimation Ø Linear regression with finite-dimensional basis expansion – Truncation of spherical wave function expansion – Estimation of expansion coefs by linear ridge regression ( ) September 3, 2024 12 <latexit sha1_base64="0OkBaoq/FquuK30YCcYUCWtno4I=">AAADsXicfVJdb9MwFHUWBqN8bfDIi0UF6thWNQUNXpAmxsN4AIpEt6G5LY7rpFYdJ7IdRDH5E/AIf4x/w03ajq2lWIp0fO65956b6zCTwthW67e35l9Zv3pt43rtxs1bt+9sbt09NmmuGe+yVKb6NKSGS6F41wor+WmmOU1CyU/C8WEZP/nMtRGp+mAnGe8lNFYiEoxaoAZb3joZUetIQu1ICxU7EiYuL4oCP3qBCdVxItRgOUyEqrgwdIdF3zXe7gTb/TZkkW+4lJgC71WAdEYC2MV80PXbeAcTCVaHFK7LLUoFIbXSh+SRbfyt158aMpE7At15lwvlgHsNTbSIR3a77/aCYkX2zO1gs95qtqqDl0EwA3U0Ox34cT/IMGV5wpVlkhpzFrQy23NUW8EkL2okNzyjbExjfgZQ0YSbnqsWVuCHwAxxlGr4lMUVezHD0cSYSRKCsrK6GCvJf8XOchs97zmhstxyxaaNolxim+Jy+3goNGdWTgBQpgV4xWxENWUW3kjtUpsw2a1WYiID07ziMKXmb4B5l3FNbaofu+p90C8FTB2T3RL9TyjUXAholRCKiER85YU7RyulQs2lcwRrDBaXtgyO281gv7n//mn94OVsoRvoPnqAGihAz9ABOkId1EXMk95376f3y3/if/Q/+eFUuubNcu6hS8cf/wE+zjmX</latexit> ˆ ˚ u = arg min ˚ u2C(N+1)2 ks ˚ uk2 + k˚ uk2 = H + I 1 Hs <latexit sha1_base64="UTeURbIWpT12en/b/r7Bpp0fQFY=">AAADyHicfVLbbhMxEPV2uZRwa+ERHiwiqgS10QahgpAiVcADQgKK1LRFcRp5He/GZO1d+VIaXL/AJ/AVvMLX8Dd4NwnqLR1pV0dnzhx7ZhwXGVM6iv4GS+GVq9euL9+o3bx1+87dldV7uyo3ktAuybNc7sdY0YwJ2tVMZ3S/kBTzOKN78fh1md87pFKxXOzoSUH7HKeCJYxg7anBavDQNFDMrXRNuIZwUcj8CCJl+MAiYTqRO7Af3JzgprPhWc+Vf4g41iPJRGqNm5mUopLkNnfe8Uvl4hpjdDzNb1wgQ8dN+LkSrvsTvG1GE92AKJGY2MVlzl5u6o0kS0e6CRGqrb3swFKFDrEsRmx+3ebBtEYldsedbKhMG+cGK/WoFVUBz4P2DNTBLLb9PH+iYU4Mp0KTDCvVa0eF7lssNSMZdTVkFC0wGeOU9jwUmFPVt9UeHXzsmSFMcuk/oWHFnqywmCs14bFXVrc+myvJi3I9o5MXfctEYTQVZHpQYjKoc1g+CjhkkhKdTTzARDJ/V0hG2M9f+6dTO3VMzNerKalE+W7eUN+lpO8987GgEutcPrEIy5TjI+e7TtF6iS4TMjEXerRI6E0YZ9+os//RQikTc+kc+TW2zy7tPNh92mpvtjY/PatvvZotdBk8AI9AA7TBc7AF3oJt0AUk+BH8Cn4Hf8J3YRF+DSdT6VIwq7kPTkX4/R8XAkgu</latexit> u(r) ⇡ N X ⌫=0 ⌫ X µ= ⌫ ˚ u(ro)j⌫(kkr ro k)Y⌫,µ ✓ r ro kr ro k ◆ := '(r)T˚ u Basis function vector Expansion coef vector <latexit sha1_base64="FiFJVDnI/rcRBQL0/t8bc+E+oyY=">AAAC+3icfZHLbhMxFIad4VbCLS1LNhYRUouiaKZChQ1SBSzYVASpaStlhsjjnEms+jKyPVWD5afgEbqr2MK7ILGF58AzTRC9cSRL3/nP78vxyUvOjI3jH63oxs1bt++s3G3fu//g4aPO6tqeUZWmMKSKK32QEwOcSRhaZjkclBqIyDns54dv6/r+EWjDlNy18xIyQaaSFYwSG6RxZ5DmwqWDGfP4NR41yRHRZcjX60T7cbLRwymfKGt6V5V3NrJPLhXEzkzhdr0fd7pxP24CX4ZkAV20iMF4tfUlnShaCZCWcmLMKIlLmzmiLaMcfDutDJSEHpIpjAJKIsBkrmnd42dBmeBC6bCkxY367w5HhDFzkQdn88aLtVq8qjaqbPEqc0yWlQVJzy4qKo6twvU/4gnTQC2fByBUs/BWTGdEE2rDb7fPXZOLXn2+NoUJ3byD0KWGnaB8KEETq/RzlxI9FeTYh66naa+m/xmZXBoDXWcMhzDBPoN3f+laK5NL65LCGJOLQ7sMe5v9ZKu/9fFFd/vNYqAr6Al6itZRgl6ibfQeDdAQUfQd/US/0O/IRyfRafT1zBq1Fnseo3MRffsDvn34zA==</latexit> = ['(r1), . . . , '(rM )]T Truncation/discretization is necessary for constructing basis functions
13. Kernel regression for sound field esitmation Ø Problem to be solved September 3, 2024 13 Kernel ridge regression with constraint that the interpolated function satisfies Helmholtz eq Solution space of Helmholtz eq § If is properly defined as reproducing kernel Hilbert space (RKHS), this problem has closed-form solution § Kernel regression can be regarded as infinite-dimensional basis expansion Good performance without truncation/discretization <latexit sha1_base64="CCNheoHdwk+3+XQETj9IXlSAYTs=">AAACtHicfZHbSsNAEIa38VzPeulNsAgiUhKR6qWoF96IClaFpshkO6mLewi7G7GGPIV4q8/l27iprXgeWPj459/ZmZ045czYIHiteCOjY+MTk1PV6ZnZufmFxaULozJNsUkVV/oqBoOcSWxaZjlepRpBxBwv49uDMn95h9owJc9tL8W2gK5kCaNgndSKBNgbQ3V+VFwv1IJ60A//J4QDqJFBnF4vVh6jjqKZQGkpB2NaYZDadg7aMsqxqEaZwRToLXSx5VCCQNPO+z0X/ppTOn6itDvS+n31840chDE9ETtnv8fvuVL8LdfKbLLbzplMM4uSvj+UZNy3yi8/wO8wjdTyngOgmrlefXoDGqh131T98kwsNsv62iTGTXOIbkqNx045SVGDVXojj0B3BdwXbuputFnSf0Ymh0ZHfxldESbYAxb5B/1pZXJoHZJbY/h9aT/hYqseNuqNs+3a3v5goZNkhaySdRKSHbJHjsgpaRJKFHkiz+TFa3iRRz18t3qVwZ1l8iU8+QacaNwN</latexit> H [Ueno+ 2018, 2021, Koyama+ ICASSP 2022 Tutorial]
14. Kernel regression for sound field esitmation Ø Unique solution with closed-form for RKHS – Based on representer theorem, the solution is represented by weighted sum of reproducing kernel function : – Vector of is obtained by with September 3, 2024 14 Estimation is achieved by convoluting FIR filter in time domain <latexit sha1_base64="zt46LX9gSaZcNDUXQVGZZLwj3Zc=">AAADdnicjZHditNAFMcnjR9r/OrqpSCDZXVXSklEVm+ERb0QpLiC3S40pZ5MTrNDM5MwM1m2hjyFj+UT+CZeOmmbxXateGDgx//858w5c6I85dr4/k+n5V67fuPmzi3v9p279+63dx+c6KxQDAcsSzN1GoHGlEscGG5SPM0VgohSHEazd3V+eI5K80x+MfMcxwISyaecgbHSpP0jjET5sfLeUC+MMOGyjAQYxS8qL5xBnsN+bVDVJOjShg7oUxqyODO6hi2u/gENQy88b2xxA0ulzq3d7P9X/f5mfZTxZceTdsfv+YugVyFYQYes4niy63wP44wVAqVhKWg9CvzcjEtQhrMU7RcUGnNgM0hwZFGCQD0uF79e0T2rxHSaKXukoQv1zxslCK3nIrJO2+CZ3szV4t9yo8JMX49LLvPCoGTLh6ZFSk1G6xXSmCtkJp1bAKa47ZWyM1DAjF20t/ZMJLp1faWn2k7zHu2UCvtW+ZSjApOp52UIKhFwUdmpk7Bb07+MXDZGS9uMtggX/BtW5SVttXLZWBuyaww2l3YVTl70gsPe4eeXnaO3q4XukEfkCdknAXlFjsgHckwGhDmBM3S+OtD65T5299xnS2vLWd15SNbC9X8DK8EdQA==</latexit> K = 2 6 4 (r1, r1) · · · (r1, rM ) . . . ... . . . (rM , r1) · · · (rM , rM ) 3 7 5 : Gram matrix
15. Kernel regression for sound field esitmation Ø RKHS based on plane wave expansion Ø Inner product and norm over using directional weighting September 3, 2024 15 How to design RKHS? Prior information on directions of high amplitude (e.g., source directions) can be incorporated <latexit sha1_base64="isRGdCdgAyvToZm2G6zcexSm7tk=">AAADfnicfVFbb9MwGHUWLqPcOnjkxaICdagrTTWN8YA0DR72AGIIuk2qS+S4X1LT2IlsByjGv4KfxhP/Bjdrq10YnxTp5Hznu/kkZc616fX+BGvhtes3bq7faty+c/fe/ebGgyNdVIrBgBV5oU4SqiHnEgaGmxxOSgVUJDkcJ9PX8/zxV1CaF/KTmZUwEjSTPOWMGk/Fzd9EUDPRTNkDh19hkkNqiMUkVZTZyNltUnKHCZcmtrU0SexHF/c9Z3g+Blu5NkmEJWCo28S1RAkL7rPdWv58cVPfmMosB7zSdmqofB9VZ9yqduxWIkww2edZ9nMOlvPm2+C3cb99dp8O/rZJFM8mhri42ep1e3XgyyBagBZaxGG8Efwi44JVAqRhOdV6GPVKM7JUGc78ag1SaSgpm9IMhh5KKkCPbP36Dj/xzBinhfKfNLhmz1ZYKrSeicQr67e+mJuT/8oNK5PujiyXZWVAstNBaZVjU+C5lXjMFTCTzzygTHG/K2YT6o0z3vDGuTGJ6NSvq1Ptr3kD/koF7zzzvgRFTaGeWUJVJuh356/OSGeO/ifkcin06Cqhb8IF/wHOrtCVUi6X0iXyNkYXTbsMjvrdaKe782G7tbe/MHQdPUKPURtF6AXaQwfoEA0QC14GcTAJeIjCp+FW+PxUuhYsah6icxHu/gUg4iX8</latexit> H = ( 1 4⇡ Z S 2 ˜ u(⌘)e jkh⌘,rid⌘ ˜ u 2 L2(S 2, w) )
16. Kernel regression for sound field esitmation Ø Kernel function for based on von Mises‒Fisher distribution Ø When no prior information, i.e., uniform weight , September 3, 2024 16 How to design RKHS? with
17. Kernel regression for sound field esitmation Ø Experimental results using real data from MeshRIR dataset – Reconstructing pulse signal from single loudspeaker w/ 18 mic September 3, 2024 17 True Proposed Gaussian kernel (Black dots indicate mic positions) Impulse response measurement system [Koyama+ 2021]
18. Neural networks for sound field estimation Ø Previous techniques based on linear and kernel regressions – Prior information on source directions can be incorporated, but the estimator is manually designed and fixed during estimation – Estimator adaptive to acoustic environment where the measurement is performed will have more flexibility and high accuracy September 3, 2024 18 High representational power and adaptability to observations/datasets of neural networks will be useful for this purpose Input Output How to embed physical properties?
19. PINN for sound field estimation Ø Physics-informed neural network (PINN) [Raissi+ 2019] – Based on implicit neural representation (or neural field) to implicitly represent a continuous function by neural network – Loss function penalizing deviation from governing eq (or PDE loss/physics loss) is added to data loss/observation loss September 3, 2024 19 Input Output <latexit sha1_base64="upivb77T/u0SHof6imIve9fNe3o=">AAACznicfZFNbxMxEIad5asNXy0cuayIkAqqol2EChKXCjhwoQSJtJWyUTTrzG6s2t6VPVsRLIsrnPkhXMtP4d/gTRNEW8pIlh+98/pjZvJaCktJ8qsTXbl67fqNtfXuzVu379zd2Ly3b6vGcBzySlbmMAeLUmgckiCJh7VBULnEg/zodZs/OEZjRaU/0rzGsYJSi0JwoCBNNnrlVpYrZ/zLdstohgR+4jIFNDPK7e15/zi4kn6yiPgipEvosWUMJpud79m04o1CTVyCtaM0qWnswJDgEn03ayzWwI+gxFFADQrt2C2q8fGjoEzjojJhaYoX6t8nHChr5yoPzvaX9nyuFf+VGzVUvBg7oeuGUPPTh4pGxlTFbWviqTDISc4DADci/DXmMzDAKTSwe+aZXG0vOmQLG6p5g6FKg++C8r5GA1SZJy4DUyr45EPVZbbd0v+MQq+MgS4zhkuEEp/Ruz90qVXolXVFYYzp+aFdhP2n/XSnv/PhWW/31XKga+wBe8i2WMqes132lg3YkHH2jf1gJ+xnNIiOIx99ObVGneWZ++xMRF9/A92F5o0=</latexit> g(r; ✓NN) : Function represented by neural network
20. PINN for sound field estimation Ø Physics-informed neural network (PINN) [Raissi+ 2019] September 3, 2024 20 <latexit sha1_base64="/KSDW3J6TWfvx2FKerzDUKs9b8g=">AAADmXicjVLbbhMxEPV2uZRwaQqPfbGIkFJaomyECgJVKpeHUqkhiKatlE1WXsdJrNjele1FpI6f4A/4On6Cb8C72aJeCGIkS8cz54xnPBOnjCrdbP70VvwbN2/dXr1TuXvv/oO16vrDY5VkEpMuTlgiT2OkCKOCdDXVjJymkiAeM3IST9/l8ZMvRCqaiCM9S0mfo7GgI4qRdq6o+ivkVFBOz0hkwpibUE+IRtZdONITyU27ba0tLhgxc3Ah0vmQx+pLVZvw1S4MVcYjw3cDOziEcxVx+AyOC420EX+9XDwftOBWKRe5vA3n9VCgmKFFpdLaQWtrOmht/l9CmGeMqrVmo1kYvA6CEtRAaZ1o3fsRDhOccSI0ZkipXtBMdd8gqSlmxFbCTJEU4Skak56DAnGi+qYYjIVPnGcIR4l0R2hYeC8qDOJKzXjsmHmp6mosd/4t1sv06GXfUJFmmgi8eGiUMagTmE8ZDqkkWLOZAwhL6mqFeIIkwtrtQuXSMzHfLr5JjZTr5j1xXUpy6DwfUyKRTuRTEyI55uirdV2Pw+0c/YtIxTnRoWVEl6TYOmv+oKXUckEd9Ry5MQZXh3YdHLcawU5j59Pz2t7bcqCrYAM8BnUQgBdgD+yDDugC7B15Z94377u/4b/x9/2DBXXFKzWPwCXzP/8GU9syEg==</latexit> minimize ✓NN JPINN(✓NN) := M X m=1 |sm g(rm; ✓NN)|2 + N X n=1 |(r2 r + k2)g(rm; ✓NN)|2 <latexit sha1_base64="zCg6p69HjmrJB4GrY+2kRUfUKOo=">AAACxnicfZHdShtBFMcnq21t+mGsl94shkIpEnZFrJdivZBCqQWjQjaEs5OzcXBmdpk5axuHhfYZfIjeto/Tt+nsmkjV2gMDv/mf/3ycc9JCCktR9LsVLCw+evxk6Wn72fMXL5c7K6+ObV4ajn2ey9ycpmBRCo19EiTxtDAIKpV4kp6/r/MnF2isyPURTQscKphokQkO5KVRZy1RQGccpPtQjVyzMcqNgaCqRp1u1IuaCO9DPIMum8XhaKV1lYxzXirUxCVYO4ijgoYODAkusWonpcUC+DlMcOBRg0I7dE0VVfjaK+Mwy41fmsJG/fuEA2XtVKXeWX/T3s3V4r9yg5KynaETuigJNb9+KCtlSHlYtyQcC4Oc5NQDcCP8X0N+BgY4+ca1bz2Tqo2mRTazvpp99FUa/OiVTwUaoNy8dQmYiYKvla96kmzU9D+j0HOjp4eM/hKhxCVW7oYetAo9t87JjzG+O7T7cLzZi7d725+3urt7s4EusTW2zt6wmL1ju+yAHbI+4+w7+8F+sl/BQaCDMvhybQ1aszOr7FYE3/4AShLjlg==</latexit> Jdata <latexit sha1_base64="w7F/E1I+85w8BY154v47c18Ny5s=">AAACxXicfZHdahNBFMcn61eNH031Tm8WgyBSwm4ptZdFKy0FMYJpC9kQzk7OpkNnZpeZs6VxWPQdfIfe6uv0bTq7TcS21gMDv/mf/3ycc9JCCktRdN4K7ty9d//B0sP2o8dPni53Vp7t27w0HAc8l7k5TMGiFBoHJEjiYWEQVCrxID3+UOcPTtBYkeuvNCtwpGCqRSY4kJfGnReJAjriIN1eNXbNxijX3/5YVeNON+pFTYQ3IZ5Dl82jP15p/UwmOS8VauISrB3GUUEjB4YEl1i1k9JiAfwYpjj0qEGhHbmmiCp87ZVJmOXGL01ho/59woGydqZS76x/aa/navFfuWFJ2ebICV2UhJpfPpSVMqQ8rDsSToRBTnLmAbgR/q8hPwIDnHzf2leeSdVq0yGbWV/NNvoqDX7yyucCDVBu3roEzFTBaeWrniarNf3PKPTC6Ok2o79EKPENK/eHbrUKvbAuyI8xvj60m7C/1os3ehtf1rtb7+cDXWIv2Sv2hsXsHdtiu6zPBoyzH+yM/WK/g51ABRScXFqD1vzMc3Ylgu8XYYXiyw==</latexit> JPDE Ground truth Reconstruction from 33 ch PINN NN [Pezzoli+ 2023]
21. Feedforward neural network for sound field estimation Ø Drawbacks of PINN – Basically, single array observation is used for training and inference, i.e., training data set is not used – Estimate does not strictly satisfy Helmholtz/wave eq because of penalization by PDE loss September 3, 2024 21 How can physical properties be embedded in neural networks that have discrete output values? Input Output
22. Feedforward neural network for sound field estimation Ø Estimating expansion coefs of basis expansion using neural networks – Train a model estimating expansion coefs of basis expansion – Continuous function can be reconstructed by using estimated expansion coefs – Can be regarded as physics-constrained neural network (PCNN) [Karakonstantis+ 2023, Lobato+ 2024] Ø Approximate PDE loss – Because of discrete output values, computation of PDE loss is not straightforward – Approximate PDE loss can be computed by finite difference or interpolation – In [Shigemi+ 2022], physics-informed convolutional neural network (PICNN) using bi- cubic interpolation is proposed September 3, 2024 22
23. Neural kernel for sound field estimation Ø Kernel function with constrainst of Helmholtz eq is optimized to acoustic environment with the aid of neural networks [Ribeiro+ 2023] – Superposition of two kernel functions – Directed kernel: direct source and early reflections – Residual kernel: late reverberations and residual components September 3, 2024 23 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="0rmx5Ei2B3tRv8KkicAKVISG2OQ=">AAAC4XicfZHNThsxEMedLW1p+hXKsReLqFLVomi3qgKXSqjl0EsFSASQslE068wGK7bXsr2IdLUP0FsFR3iaXssL8DZ481EVUjqSpZ/+87fHM5Nowa0Lw+ta8GDp4aPHy0/qT589f/GysfLqwGa5YdhhmcjMUQIWBVfYcdwJPNIGQSYCD5PRlyp/eILG8kztu7HGnoSh4iln4LzUb7TjEWgN9BOdQr+IJbhjI4sBN2VJ3y/oBm1Z9hvNsBVOgi5CNIMmmcVuf6V2Hg8ylktUjgmwthuF2vUKMI4zgWU9zi1qYCMYYtejAom2V0waLOkbrwxomhl/lKMT9e8bBUhrxzLxzuqX9m6uEv+V6+Yu3ewVXOncoWLTQmkuqMtoNS3qh4DMibEHYIb7v1J2DAaY8zOt3yqTyPXJhGxqfTfb6Ls0+M0rOxoNuMy8K2IwQwmnpe96GK9X9D8jV3Ojp/uM/hEu+Xcsiz90r5WruXVOfo3R3aUtwsGHVtRutfc+Nrc+zxa6TF6TNfKWRGSDbJGvZJd0CCOX5Bf5Ta4CFvwIfgZnU2tQm91ZJbciuLgBJGTucg==</latexit>  = dir + res Directed kernel Residual kernel
24. Neural kernel for sound field estimation Ø Directed kernel – Directional weighting with weighted sum of (sparse) von Mises—Fisher distribution [Horiuchi+ 2021] September 3, 2024 24 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="I2jQ2Fgmjq+z5rZFxQfy+e9Ar9o=">AAADQ3icfZHNbtQwEMed8FWWj27hyMViQdqiZZWgqkVClSrKgQtQJLattNlGjneSWo2dyHaAxfJTcOGFeAiegRviCsJJdle0pYwU6af//GecmUnKnCkdBN88/9LlK1evrVzv3Lh56/Zqd+3OvioqSWFEi7yQhwlRkDMBI810DoelBMKTHA6Sk906f/AepGKFeKdnJUw4yQRLGSXaSXH3y4fYRJzoY8nNlElr+1HCTQSa2Ge4waTmQcsZ4ZzYdbyNI1Xx2Ijt0B69xq0eCxylktBlP7BHbXmdyYnIcsDL7m3Hqa1zsslZa3b7c/+6jbu9YBg0gc9DOIcemsdevOZ9jqYFrTgITXOi1DgMSj0xRGpGXfdOVCkoCT0hGYwdCsJBTUyzQosfOmWK00K6T2jcqH9XGMKVmvHEOevp1NlcLf4rN650+nRimCgrDYK2D6VVjnWB63tgt3SgOp85IFQy96+YHhO3Ru2u1jn1TMIHzWZVqtw0L8BNKeGVU96UIIku5CMTEZlx8tG6qbNoUNP/jEwsjI4uMromjLNPYM2SLrQysbAuyJ0xPHu087D/ZBhuDjffbvR2ns8PuoLuofuoj0K0hXbQS7SHRoii394D77E39L/63/0f/s/W6nvzmrvoVPi//gCenhPw</latexit> wdir(⌘; , ) = N X n=1 n e n h⌘,dn i C( n) <latexit sha1_base64="wce9c2fonwWBBzEGuOJL2nk9Ius=">AAACwHicfZFbSxtBFMcnq201vUXFJ18WQ8EWCbulaF8EUR98ERUaFbIhnJ2c3QzOZZ2ZLcZ1P4WfwFf9RH4bZ2Mi9dYDAz/+5z8z5xJnnBkbBHc1b2r63fsPM7P1j58+f/namJs/MirXFNtUcaVPYjDImcS2ZZbjSaYRRMzxOD7drvLHf1EbpuQfO8ywKyCVLGEUrJN6jcWV6DKKRRGlIASU0WUv3Ai/9xrNoBWMwn8J4RiaZBwHvbnaVdRXNBcoLeVgTCcMMtstQFtGOZb1KDeYAT2FFDsOJQg03WJUf+l/c0rfT5R2R1p/pP57owBhzFDEzinADszzXCW+luvkNvndLZjMcouSPnyU5Ny3yq+G4feZRmr50AFQzVytPh2ABmrdyOpPvonFavW+Nolx3eyg61LjnlP2M9Rglf5RRKBTAeel6zqNViv6n5HJidHRW0b3CBPsAsvikd60MjmxTsitMXy+tJdw9LMVrrXWDn81N7fGC50hS2SZrJCQrJNNsksOSJtQUpBrckNuvS1v4Cnv7MHq1cZ3FsiT8C7uAZQE35U=</latexit> (k k1 = 1) <latexit sha1_base64="44H961efgdVBY4JkEGr9+jx5ReM=">AAADfHiclZFdb9MwFIadlo9RPtaNS24sClILXZVUqONm0sS44AYY0rpNqrvIcZ3Ua+xktoMoln8Fv4xL/gzCSVPENobEkSI9Ouc95+T4jfKUKe37P7xG89btO3c37rXuP3j4aLO9tX2sskISOiZZmsnTCCuaMkHHmumUnuaSYh6l9CRaHJT1k89UKpaJI73M6ZTjRLCYEaxdKmx/Rwuc5zg0iGM9l9zMmLS2iyJupA2Dfg3DHtyDSBU8NGIvsGcfIEow5zgUsIViiYk5D32U0lh3nexCagO764nnFkVUY1iOQg4s3IGLeq4JhrZ3tlquYnNk7X/0QYskS+a6Z81Bt9KGomfDdscf+FXA6xDU0AF1HIZb3jc0y0jBqdAkxUpNAj/XU4OlZiSltoUKRXNMFjihE4cCc6qmpnp7C5+7zAzGmXSf0LDK/tlhMFdqySOnrI68WiuTf6tNCh2/nhom8kJTQVaL4iKFOoOlkdAZRYlOlw4wkcz9KyRz7KzQzu7WpTUR71dvqmLlrnlL3ZWSvneZjzmVWGfyhUFYJhx/se7qBPVL+peQibXQ0U1CN4Rx9pVa85tulDKxlq7J2RhcNe06HA8HwWgw+vSqs/+mNnQDPAFPQRcEYBfsg3fgEIwB8Xa9qRd7SeNn81nzZXNnJW14dc9jcCmao1839ST1</latexit> dir(r1, r2) = N X n=1 n j0 ⇣p (j ⌘ kr12)T(j ⌘ kr12) ⌘ C( n) Sparsity constraint Normalization constant
25. Neural kernel for sound field estimation Ø Residual kernel – Directional weighting with implicit neural representation September 3, 2024 25 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="s4P1x3nuyJ2mvTCbZ2ZsvG/WxFE=">AAAC+XicfZHdahNBFMcn61eNX6leejMYhCol7BaphSIU9cIbYwXTFrIhnJ2cTYbuzC4zZ9U47FP4BN6Jt/oyeqsP4uw2EdsaDwz85n/+83HOSYpMWgrD763gwsVLl6+sXW1fu37j5q3O+u0Dm5dG4EDkWW6OErCYSY0DkpThUWEQVJLhYXL8rM4fvkVjZa7f0LzAkYKplqkUQF4ad/rvxi5WQDOjnEFbVRtxolyMBNUub5Bm9eYBf8KXvn5/pWvc6Ya9sAl+HqIFdNki9sfrrY/xJBelQk0iA2uHUVjQyIEhKTKs2nFpsQBxDFMcetSg0I5cU3jF73tlwtPc+KWJN+rfJxwoa+cq8c767/Zsrhb/lRuWlO6MnNRFSajFyUNpmXHKed1FPpEGBWVzDyCM9H/lYgYGBPlet089k6jNpm82tb6a5+irNPjSK68KNEC5eehiMFMF7ytf9TTerOl/RqmXRk+rjP4SqeQHrNwfWmmVemldkh9jdHZo5+Fgqxdt97ZfP+ruPV0MdI3dZffYBovYY7bHXrB9NmCCfWM/2E/2K3DBp+Bz8OXEGrQWZ+6wUxF8/Q016fgt</latexit> wres(⌘; ✓) = NN(⌘; ✓) <latexit sha1_base64="dexZadk2Poc298GGiqVC2ayTShI=">AAADX3icfZHfihMxFMYzrbpr1bWrV+pNsAi7UktnkVUQoagXKogr2t2FppYz6WkbO8kMSUatQ57CV/FlvPRNzEynxf3ngYHffOc7OTk5URoLY7vd30GtfunylY3Nq41r129s3Wxu3zo0SaY59nkSJ/o4AoOxUNi3wsZ4nGoEGcV4FM1fFvmjr6iNSNQnu0hxKGGqxERwsF4aNX+xOaQpjHImwc60zDUa53ZY5MmNwnYFe7v0OWVC2coYRflHrzr67bxKhhbcs5LsrOBdujKh+5w/Wv18cSwGNY2RrqvadNmRMl1m3Lpy7NamUbPV7XTLoGchrKBFqjgYbQc/2TjhmURleQzGDMJuaoc5aCu4b9JgmcEU+BymOPCoQKIZ5uXzOvrAK2M6SbT/lKWl+m9FDtKYhYy8s7irOZ0rxPNyg8xOng5zodLMouLLRpMspjahxa7oWGjkNl54AK6FvyvlM9DArd9o40SbSLbLdzIT46d5hX5Kje+88j5FDTbRD3MGeirhu/NTT1m7oP8ZhVoZPV1k9IcIKX6gy9d0oVWolXVFfo3h6aWdhcO9Trjf2f/wuNV7US10k9wj98kOCckT0iOvyQHpEx7cDXrBm+Bt7U99o75Vby6ttaCquU1ORP3OX+2/HOQ=</latexit> res(r1, r2) = Z S 2 wres(⌘; ✓)e jh⌘,rid⌘ Computed by numerical integration : Implicit neural representation
26. Neural kernel for sound field estimation Ø Again, (positive-definite) kernel function is the sum of directed and residual kernels – Hyperparameters are jointly optimized by a steepest-descent-based algorithm – Physics-constraint is still preserved – Estimation process is still linear operation in freq domain based on kernel ridge regression September 3, 2024 26 Reproducing kernel function adapted to acoustic environment using neural networks <latexit sha1_base64="0rmx5Ei2B3tRv8KkicAKVISG2OQ=">AAAC4XicfZHNThsxEMedLW1p+hXKsReLqFLVomi3qgKXSqjl0EsFSASQslE068wGK7bXsr2IdLUP0FsFR3iaXssL8DZ481EVUjqSpZ/+87fHM5Nowa0Lw+ta8GDp4aPHy0/qT589f/GysfLqwGa5YdhhmcjMUQIWBVfYcdwJPNIGQSYCD5PRlyp/eILG8kztu7HGnoSh4iln4LzUb7TjEWgN9BOdQr+IJbhjI4sBN2VJ3y/oBm1Z9hvNsBVOgi5CNIMmmcVuf6V2Hg8ylktUjgmwthuF2vUKMI4zgWU9zi1qYCMYYtejAom2V0waLOkbrwxomhl/lKMT9e8bBUhrxzLxzuqX9m6uEv+V6+Yu3ewVXOncoWLTQmkuqMtoNS3qh4DMibEHYIb7v1J2DAaY8zOt3yqTyPXJhGxqfTfb6Ls0+M0rOxoNuMy8K2IwQwmnpe96GK9X9D8jV3Ojp/uM/hEu+Xcsiz90r5WruXVOfo3R3aUtwsGHVtRutfc+Nrc+zxa6TF6TNfKWRGSDbJGvZJd0CCOX5Bf5Ta4CFvwIfgZnU2tQm91ZJbciuLgBJGTucg==</latexit>  = dir + res Directed kernel Residual kernel <latexit sha1_base64="LfIE5/umsVmKZKr2rKhL6DRhj+4=">AAACz3icfZFNbxMxEIad5auEj6Zw5GIRkBCKot2qKhwr6IELaiuRtlI2imad2Y1V27uyZ4GwWtQrXPkjXOk/6b+pN00q2lJGsvT4nXdsjycplHQUhqet4NbtO3fvrdxvP3j46PFqZ+3JvstLK3AgcpXbwwQcKmlwQJIUHhYWQScKD5Kj903+4DNaJ3PziWYFjjRkRqZSAHlp3HkRJ7qKEySoe3zOGWh9saFpkxl3umE/nAe/DtECumwRu+O11q94kotSoyGhwLlhFBY0qsCSFArrdlw6LEAcQYZDjwY0ulE1b6fmL70y4Wlu/TLE5+rfFRVo52Y68U4NNHVXc434r9ywpPTtqJKmKAmNOL8oLRWnnDd/wyfSoiA18wDCSv9WLqZgQZD/wfalaxLda863LnW+m230XVr86JWdAi1Qbl9XMdhMw9fad53FvYb+Z5RmafR0k9EfIrX8hnV1QTdapVlal+THGF0d2nXYX+9Hm/3NvY3u1rvFQFfYM/acvWIRe8O22Ae2ywZMsJ/sN/vDToK94EvwPTg+twatRc1TdimCH2e3guZ0</latexit> , , ✓
27. Neural kernel for sound field estimation Ø Numerical experiment: T60: 400 ms, # mics: 41, spherical shell array September 3, 2024 27 Ground truth (600 Hz) NN PINN Adaptive kernel NMSE: -6.8 dB NMSE: -16.3 dB NMSE: -24.8dB
28. Overview article for PIML-based sound field estimation Ø Physics-Informed Machine Learning For Sound Field Estimation – To appear in IEEE Signal Processing Magazine, but preprint is available below September 3, 2024 28 https://arxiv.org/abs/2408.14731
29. Application to Binaural Reproduction September 3, 2024 29 Conversion into binaural sounds Ø Binaural reproduction in real world is difficult, compared to binaural synthesis in VR space Ø Binaural reproduction from recordings of multiple small arrays instead of single spherical array Ø Broad listening area by using flexible and scalable recording system Binaural reproduction from mic array recordings for VR audio Recording Reproduction [Iijima+ JASA 2021]
30. Application to Binaural Reproduction Ø Recording system using multiple Ambisonic mics and 360-degree cameras September 3, 2024 30 Small mic arrays (Ambisonic mics) 360-degree cameras Demo Proposed Single array [Iijima+ IEEE WASPAA 2021 (demo)] Error distribution
31. September 3, 2024 31 Headphones Recommended
32. SOUND FIELD CONTROL September 3, 2024 32
33. Sound field control problem Ø Optimization problem to be solved September 3, 2024 33 Difficult to solve owing to regional integration Goal: Synthesizing desired sound field inside with secondary sources (loudspeakers) <latexit sha1_base64="SbdBLMPsjw6ciQkjzJuiJJhjGvk=">AAACF3icbVDLSsNAFJ3UV62vqksXBotQQUoiRV0W3bisYB/QlDCZ3LZDZyZhZiKUkKUf4Te41bU7cevSpX/i9LGwrQcuHM65l3vvCWJGlXacbyu3srq2vpHfLGxt7+zuFfcPmipKJIEGiVgk2wFWwKiAhqaaQTuWgHnAoBUMb8d+6xGkopF40KMYuhz3Be1RgrWR/OJx4qcex3ogeRqCyrKyF/BUZudexKGPz/xiyak4E9jLxJ2REpqh7hd/vDAiCQehCcNKdVwn1t0US00Jg6zgJQpiTIa4Dx1DBeaguunkkcw+NUpo9yJpSmh7ov6dSDFXasQD0zm+WS16Y/E/r5Po3nU3pSJONAgyXdRLmK0je5yKHVIJRLORIZhIam61yQBLTLTJbm5LwDOTibuYwDJpXlTcy0r1vlqq3czSyaMjdILKyEVXqIbuUB01EEFP6AW9ojfr2Xq3PqzPaWvOms0cojlYX7/bXqDc</latexit> udes(r, !) <latexit sha1_base64="OlngG7bMYm0AnUElZFcGXeuuUPg=">AAAB/HicbVA9SwNBEJ2LXzF+RS1tFoNgFe5E1DJoY2cE8wHJEfY2k2TN7t2xuyeEI/4GW63txNb/Yuk/cZNcYRIfDDzem2FmXhALro3rfju5ldW19Y38ZmFre2d3r7h/UNdRohjWWCQi1QyoRsFDrBluBDZjhVQGAhvB8GbiN55QaR6FD2YUoy9pP+Q9zqixUr19J7FPO8WSW3anIMvEy0gJMlQ7xZ92N2KJxNAwQbVueW5s/JQqw5nAcaGdaIwpG9I+tiwNqUTtp9Nrx+TEKl3Si5St0JCp+ncipVLrkQxsp6RmoBe9ifif10pM78pPeRgnBkM2W9RLBDERmbxOulwhM2JkCWWK21sJG1BFmbEBzW0J5Nhm4i0msEzqZ2Xvonx+f16qXGfp5OEIjuEUPLiECtxCFWrA4BFe4BXenGfn3flwPmetOSebOYQ5OF+/In2Vmg==</latexit> ⌦ <latexit sha1_base64="VJ5RMQ2GKmQZdUQJz96dZvLRnxA=">AAAB93icbVA9SwNBEN2LXzF+RS1tFoNgFe4kqGXQxsIiAfMByRH2NnPJkt29Y3dPOI78Alut7cTWn2PpP3GTXGESHww83pthZl4Qc6aN6347hY3Nre2d4m5pb//g8Kh8fNLWUaIotGjEI9UNiAbOJLQMMxy6sQIiAg6dYHI/8zvPoDSL5JNJY/AFGUkWMkqMlZqPg3LFrbpz4HXi5aSCcjQG5Z/+MKKJAGkoJ1r3PDc2fkaUYZTDtNRPNMSETsgIepZKIkD72fzQKb6wyhCHkbIlDZ6rfycyIrRORWA7BTFjverNxP+8XmLCWz9jMk4MSLpYFCYcmwjPvsZDpoAanlpCqGL2VkzHRBFqbDZLWwIxtZl4qwmsk/ZV1buu1pq1Sv0uT6eIztA5ukQeukF19IAaqIUoAvSCXtGbkzrvzofzuWgtOPnMKVqC8/ULSPOTbw==</latexit> L : Target region Secondary source Synthesized sound field <latexit sha1_base64="jXBfTdmzlzCxe3eJ2RCk10ZxZxw=">AAACo3icfZHbSgMxEIbT9VzPeunNYhFEpOyKqJeiXggiHqtCdymz6XQbTLJLkhXrsi8geKvP5tuY1lY8DwQ+/vmTmclEKWfaeN5ryRkaHhkdG58oT05Nz8zOzS9c6SRTFGs04Ym6iUAjZxJrhhmON6lCEBHH6+h2v5u/vkOlWSIvTSfFUEAsWYtRMFa6aDZ4Y67iVb1euD/B70OF9OO0MV96DJoJzQRKQzloXfe91IQ5KMMox6IcZBpToLcQY92iBIE6zHu9Fu6KVZpuK1H2SOP21M83chBad0RknQJMW3/PdcXfcvXMtHbCnMk0Myjpe6FWxl2TuN3B3SZTSA3vWACqmO3VpW1QQI39ni9VIlGUy8EB2uEUHttCJykqMIlaywNQsYD7wg4bB+td+s/I5MBo6S+jfYQJ9oBF/kF/WpkcWAdkt+d/39VPuNqo+lvVzbPNyu5ef4/jZIksk1Xik22ySw7JKakRSmLyRJ7Ji7PiHDnnzuW71Sn17yySL+GEb9ci1JI=</latexit> dl <latexit sha1_base64="2143p1RAQMlzka1H5YkQBHUMHYo=">AAACq3icfZFLb9NAEMc3ppQS6AuOXKxGSKVqIxtVLceo7YELIqhNWxFH0XgzdlbZh7W7RgTLX4B7r/C5+DasE6dqWspIK/0085+dV5xxZmwQ/Gl4T1aerj5be9588XJ9Y3Nr+9WlUbmm2KOKK30dg0HOJPYssxyvM40gYo5X8eS0il99Q22Ykhd2muFAQCpZwihY5+qnQ74bxaLQ5bvhVitoBzPzH0JYQ4vU1h1uN35GI0VzgdJSDsb0wyCzgwK0ZZRj2YxygxnQCaTYdyhBoBkUs55L/63zjPxEafek9WfeuxkFCGOmInZKAXZs7scq579i/dwmHwYFk1luUdJ5oSTnvlV+tQB/xDRSy6cOgGrmevXpGDRQ69a0VCUWZbMZnaEbTuMnV+hzhhqs0ntFBDoV8L10w6bRfkX/EzK5EDp6TOg+YYL9wLK4pUelTC6kC3LXC+/f6iFcvm+HR+3DL4etzkl9xzXyhuyQXRKSY9IhH0mX9AglityQX+S3d+Cde1+9aC71GnXOa7JkHv4Fm6XXyw==</latexit> gl(r) • : Driving signal of th secondary sources • : Transfer function of th secondary source <latexit sha1_base64="bKgRyukOmCpfZztJEARaEMlc1nM=">AAACoXicfZFNSwMxEIbT9bt+Vo9eFosgImVXRD0W9aAHsYrVQreU2XRag0l2SbJiXfYP6FV/nP/GtLZi1ToQeHjnTWYmE8acaeN57zlnYnJqemZ2Lj+/sLi0vFJYvdFRoihWacQjVQtBI2cSq4YZjrVYIYiQ4214f9zL3z6g0iyS16YbY0NAR7I2o2CsdMmbK0Wv5PXD/Q3+AIpkEJVmIfcctCKaCJSGctC67nuxaaSgDKMcs3yQaIyB3kMH6xYlCNSNtN9p5m5apeW2I2WPNG5f/X4jBaF1V4TWKcDc6Z+5nvhXrp6Y9mEjZTJODEr6WaidcNdEbm9st8UUUsO7FoAqZnt16R0ooMZ+zkiVUGT5fHCCdjiF57bQRYwKTKS20wBUR8BjZoftBDs9+s/I5NBoaZzRPsIEe8Is/aKxViaH1iHZ7fk/d/UbbnZL/n5p73KvWD4a7HGWrJMNskV8ckDK5JRUSJVQguSFvJI3p+icORXn6tPq5AZ31shIOPUP1irTuw==</latexit> l <latexit sha1_base64="bKgRyukOmCpfZztJEARaEMlc1nM=">AAACoXicfZFNSwMxEIbT9bt+Vo9eFosgImVXRD0W9aAHsYrVQreU2XRag0l2SbJiXfYP6FV/nP/GtLZi1ToQeHjnTWYmE8acaeN57zlnYnJqemZ2Lj+/sLi0vFJYvdFRoihWacQjVQtBI2cSq4YZjrVYIYiQ4214f9zL3z6g0iyS16YbY0NAR7I2o2CsdMmbK0Wv5PXD/Q3+AIpkEJVmIfcctCKaCJSGctC67nuxaaSgDKMcs3yQaIyB3kMH6xYlCNSNtN9p5m5apeW2I2WPNG5f/X4jBaF1V4TWKcDc6Z+5nvhXrp6Y9mEjZTJODEr6WaidcNdEbm9st8UUUsO7FoAqZnt16R0ooMZ+zkiVUGT5fHCCdjiF57bQRYwKTKS20wBUR8BjZoftBDs9+s/I5NBoaZzRPsIEe8Is/aKxViaH1iHZ7fk/d/UbbnZL/n5p73KvWD4a7HGWrJMNskV8ckDK5JRUSJVQguSFvJI3p+icORXn6tPq5AZ31shIOPUP1irTuw==</latexit> l
34. Pressure matching Ø Discretize target region into control points Ø Optimization problem for pressure matching becomes simple least-squares problem September 3, 2024 34 <latexit sha1_base64="V9SycBRbehDy9zsTAA8X3hDIM1c=">AAACpnicfZFLSwMxEMfT9V2f1aOXxSKISNkVUY9FPXjxBbYVukVn0+kaTbJLkhXrsl/Ak1f9Zn4b09qK74HAj//8k5nJhAln2njea8EZGR0bn5icKk7PzM7NL5QW6zpOFcUajXmsLkLQyJnEmmGG40WiEETIsRHe7vfyjTtUmsXy3HQTbAmIJOswCsZK9eBEYASXC2Wv4vXD/Qn+AMpkEKeXpcJj0I5pKlAaykHrpu8lppWBMoxyzItBqjEBegsRNi1KEKhbWb/d3F21StvtxMoeady++vlGBkLrrgitU4C51t9zPfG3XDM1nd1WxmSSGpT0vVAn5a6J3d7sbpsppIZ3LQBVzPbq0mtQQI39oS9VQpEXi8EB2uEUHtlCJwkqMLFazwJQkYD73A4bBRs9+s/I5NBo6S+jfYQJ9oB59kF/WpkcWodkt+d/39VPqG9W/O3K1tlWubo32OMkWSYrZI34ZIdUySE5JTVCyQ15Is/kxVlzjp2a03i3OoXBnSXyJZyrN7rP1cY=</latexit> ⌦ <latexit sha1_base64="q0uECYVB1NlCDNpVOene+1Qk19c=">AAACq3icfZFLTxsxEMedpVBIeXPkYhFVggqi3QoBR0Q5cKAF1AZQsxGadSYbCz9WtheRrvYLcOcKn6vfpk5IKt4jWfpp5j+eV5IJbl0Y/q0EYx/GJz5OTlU/Tc/Mzs0vLJ5anRuGDaaFNucJWBRcYcNxJ/A8MwgyEXiWXH7rx8+u0Fiu1S/Xy7AlIVW8wxk472r+oDFdjVOkh2sX87WwHg6MvoRoCDUytOOLhcpN3NYsl6gcE2BtMwoz1yrAOM4EltU4t5gBu4QUmx4VSLStYtBzST97T5t2tPFPOTrwPs4oQFrbk4lXSnBd+zzWd74Wa+aus9MquMpyh4o9FOrkgjpN+wugbW6QOdHzAMxw3ytlXTDAnF/TkyqJLKvVeB/9cAa/+0JHGRpw2nwpYjCphOvSD5vG6316T8jVSOjpLaH/hEv+B8viP70p5WokHZG/XvT8Vi/h9Gs92qpvnmzWdveGd5wky2SFrJKIbJNdckCOSYMwosktuSP3wUbwM/gdxA/SoDLMWSJPLMB/tKXWgg==</latexit> N ( L) <latexit sha1_base64="Sfob4tqImS+gkH7eBMlgYAHh4Uo=">AAADDnicfZJNb9QwEIad8FWWj27hyMVihYT4WCVVRXusKBIcQBSJbSutd1eOd5K1ajuR7SCWkD/AiV/DDXHlzI1/wpFJmiLaUkaK9OqdZzweT5JCSeej6GcQXrh46fKVlau9a9dv3Fztr93ac3lpBYxErnJ7kHAHShoYeekVHBQWuE4U7CeHO01+/x1YJ3Pz1i8LmGieGZlKwT1as75iWhqp5QeYVSzR1bymTBrKNPeLJKl26ulLdBSknn2kDfAcicetKutp1XIWy8DVyFmZLRCcrtOHlIHntCtqTkV31h9Ew6gNelbEnRiQLnZna8EnNs9FqcF4obhz4zgq/KTi1kuhoO6x0kHBxSHPYIzScA1uUrXPUtN76Mxpmlv8jKet+3dFxbVzS50g2YzhTuca81+5cenTrUklTVF6MOKoUVoq6nPavDGdSwvCqyUKLqzEu1Kx4JYLj5s40SXRda/HngEOZ+EVNnpdgOU+tw8qxm2m+fsah83Yo0b9D5TmGER1HoiHtKuuqz/qXLT7KxA9Vri9+PSuzoq99WH8ZLjxZmOw/bTb4wq5Q+6S+yQmm2SbvCC7ZEQE+UF+BUEQhp/DL+HX8NsRGgZdzW1yIsLvvwHgg/35</latexit> minimize d2CL Gd udes 2 + ⌘kdk2 <latexit sha1_base64="i9WfaOjLrhWO59r+nVt9MC7/SLM=">AAADEHicfZHLbhMxFIY90wIlXJrCko1FhFRu0QyqgA1SBUiUBaJIpK0Up5HHOTOxantGtgcRLL8AS56GHWLLmg2vwgrPJEH0xpEsff7Pf3xsn6wS3Ngk+RXFK6sXLl5au9y5cvXa9fXuxo09U9aawYCVotQHGTUguIKB5VbAQaWBykzAfnb0osnvfwBteKne21kFI0kLxXPOqA3SuKtIJt3E42eYCMjtJm72r/yhI5LaqcndjvdzCd/HBCxtDa89JpoXU3v30D1M/VlFrVYvNR2agPF+3O0l/aQNfBrSBfTQInbHG9FnMilZLUFZJqgxwzSp7MhRbTkT4DukNlBRdkQLGAZUVIIZufZjPL4TlAnOSx2WsrhV/61wVBozk1lwtjc/mWvEs3LD2uZPR46rqrag2LxRXgtsS9z8Mp5wDcyKWQDKNA93xWxKNWU2zOJYl0z6Toe8hPA4DW9Co7cVaGpLfc8RqgtJP/rw2II8aOh/Rq6WxkDnGcMhXPJP4N1fOtfK1dK6pDC99OSsTsPeo376uL/1bqu3/XwxxzV0C91GmyhFT9A22kG7aIAY+ol+RyvRavwl/hp/i7/PrXG0qLmJjkX84w9/cP+K</latexit> d = GHG + ⌘I 1 GHudes Transfer function matrix Driving signal vector Desired pressure vector Regularization term Closed-form solution is obtained as J Simple implementation L Fine discretization of is necessary <latexit sha1_base64="V9SycBRbehDy9zsTAA8X3hDIM1c=">AAACpnicfZFLSwMxEMfT9V2f1aOXxSKISNkVUY9FPXjxBbYVukVn0+kaTbJLkhXrsl/Ak1f9Zn4b09qK74HAj//8k5nJhAln2njea8EZGR0bn5icKk7PzM7NL5QW6zpOFcUajXmsLkLQyJnEmmGG40WiEETIsRHe7vfyjTtUmsXy3HQTbAmIJOswCsZK9eBEYASXC2Wv4vXD/Qn+AMpkEKeXpcJj0I5pKlAaykHrpu8lppWBMoxyzItBqjEBegsRNi1KEKhbWb/d3F21StvtxMoeady++vlGBkLrrgitU4C51t9zPfG3XDM1nd1WxmSSGpT0vVAn5a6J3d7sbpsppIZ3LQBVzPbq0mtQQI39oS9VQpEXi8EB2uEUHtlCJwkqMLFazwJQkYD73A4bBRs9+s/I5NBo6S+jfYQJ9oB59kF/WpkcWodkt+d/39VPqG9W/O3K1tlWubo32OMkWSYrZI34ZIdUySE5JTVCyQ15Is/kxVlzjp2a03i3OoXBnSXyJZyrN7rP1cY=</latexit> ⌦ : Target region Secondary source Control points
35. Weighted Pressure Matching Ø Original cost function is approximated as Ø Driving signals are obtained as weighted least squares solution September 3, 2024 35 Pressure matching for continuous region based on kernel regression of sound field <latexit sha1_base64="wBoAciZ2FK9Swr355ix7ob1QUvE=">AAADEnicfZHLbtNAFIYn5lbCLYUlmxERUkEosqsK2CBVLQs2qEVqmoo4RMeTY3dUz9iaGVdNR/MWPADPwQrElhfgBdjCIzB2EkR64UiWP/3nn9t/kjLn2oThj1Zw5eq16zdWbrZv3b5z915n9f6+LirFsM+KvFAHCWjMucS+4SbHg1IhiCTHQXK0XfcHx6g0L+SemZY4EpBJnnIGxkvjzvs4EXbg6Csac2nGNt4RmIGjtXzq1uqfck8+2Bi0uUAVYA51avec79WshJ3MbMqNO92wFzZFz0M0hy6Z1+54tfUpnhSsEigNy0HrYRSWZmRBGc5ydO240lgCO4IMhx4lCNQj24Tg6GOvTGhaKP9JQxv13xUWhNZTkXhnc+uzvVq8qDesTPpyZLksK4OSzQ5Kq5yagtaJ0glXyEw+9QBMcX9Xyg5BATM+96VTErH0BtskplPt2u34NfonK3zrpZ0SFZhCPfWpq0zAifMRZPGzmv5n5HJh9HSZ0W/CBT9FZ//SpVYuF9YF+ZlGZyd4HvbXe9Hz3sa7je7m1ny6K+QheUTWSERekE3yhuySPmHkC/lJfpHfwcfgc/A1+DazBq35mgdkqYLvfwAs/wN9</latexit> W = Z ⌦ z(r)⇤z(r)Tdr <latexit sha1_base64="ots8Wmdw3c6kEAqnCoYqjjAHV2E=">AAAD0HiclVJbb9MwFHYWLqNctsEjL9YqpgJb1UwT8II0ARIDhDbQuk6au8pxT1KrcRLZDmrJLMQjvPM7eIWfwr/BSdOKbhSJIyX+9J3v3HzspxFXutX65Sy5ly5fubp8rXb9xs1bK6trt49UkkkGbZZEiTz2qYKIx9DWXEdwnEqgwo+g4w+fF/7OB5CKJ/GhHqfQFTSMecAZ1ZbqrTnrr/EGoWkqkxEmPNa9nOwLCKnBJIJAn2Hii5wMrYKaRoGluX+aE0H1QAX5oal0jVL3xjwkI16gV5aXPBxoK97y5kQvTfHvm63iyEyVTFoKlJmFVefZ6Tae+c2kPiaktvG0SllmXJRs1sK0372igFV2zH+E91brrWarNHwReBWoo8oO7K1+I/2EZQJizSKq1InXSnU3p1JzFoGpkUxBStmQhnBiYUwFqG5ebtPge5bp4yCR9os1Ltk/I3IqlBoL3yrLqc77CvJvvpNMB0+6OY/TTEPMJoWCLMI6wcXTwH0ugelobAFlktteMRtQSZm2D6g2V8YXm+VFqUAZ63kBdkwJby21n4KkOpEPckJlKOjI2LFDslmgfwl5PBVatEhok3DBP4LJZ2ihlMdT6RTV7CK982u7CI62m96j5s67nfrus2qly+guWkcN5KHHaBftoQPURsz54nx3fjg/3ffuyP3kfp5Il5wq5g6aM/frbwuZR80=</latexit> J ⇡ Z ⌦ (r)T (K + ⇠I) 1 Gd udes 2 dr = Gd udes H W Gd udes <latexit sha1_base64="UqfxQe3UGsqqnmcvxH4VBmwhEN8=">AAAC/XicfZFdaxNBFIYn61eNH03rpTeDQYhaQ1aKFkEo6oUiYoSmLWRjODs5uxkyM7vMzErTZfBX+BO8Ey/1twje6u9wdpuIba0HBh7e952PMyfOBTe21/veCM6dv3Dx0srl5pWr166vttbWd01WaIYDlolM78dgUHCFA8utwP1cI8hY4F48e1b5e+9RG56pHTvPcSQhVTzhDKyXxq1+FMvy0NHHT2hF0QzyHFynYu3uvCsjCXZqknLHORoJTGynzr1y92h0wCt86Q3N06n16fuhG7favW6vLnoawgW0yaL647XGx2iSsUKiskyAMcOwl9tRCdpyJtA1o8JgDmwGKQ49KpBoRmXduqO3vTKhSab9UpbW6t87SpDGzGXsk3UnJ71K/Jc3LGyyNSq5yguLih1dlBSC2oxW/0gnXCOzYu4BmOb+rZRNQQOz/rebx66J5UZ1vjaJcd55jr5Nja+99CZHDTbTd8sIdCrhwPm202ijov8FuVoGPZ0V9IdwyQ/RlX/ozChXy+iSmn6Q4cmxnYbdB93wYXfz7WZ7++lipCvkJrlFOiQkj8g2eUH6ZEAY+Up+kJ/kV/Ah+BR8Dr4cRYPGYs8NcqyCb78Bcfj4Dw==</latexit> z := (r)T (K + ⇠I) 1 Kernel ridge regression <latexit sha1_base64="CRJvQ0OuNgbbSk0jTZw5dDlZAo8=">AAADxHiclVJbaxNBFJ7teqnx0ovgiy+DQanahqyU6otQrNgKihVMU8ikYXZyNhk6M7vMzIpxGf+Fv8BX/UH+G2e3G0nbRPDAwrff+c5tzokzwY1tt38HS+GVq9euL99o3Lx1+87K6tr6kUlzzaDDUpHq45gaEFxBx3Ir4DjTQGUsoBuf7pX+7mfQhqfqk51k0Jd0pHjCGbWeGqwF98iY2oLEshg6hx+9xITqkeRqUHOEK0wkteM4LvbcyTuHiYDEbuDSvT90eKtCuTspKpn2QWB8KqL5aGwf17RJioOS9NLu/6bAhDTKxmaC3Jy0+w4/xQQsLf/ezjSwFbnFUW5+7cFqs91qV4Yvg6gGTVTboX/J72SYslyCskxQY3pRO7P9gmrLmQDXILmBjLJTOoKeh4pKMP2i2qDDDz0zxEmq/acsrtjZiIJKYyYy9spqgIu+kpzn6+U2edEvuMpyC4qdFUpygW2Ky3PAQ66BWTHxgDLNfa+YjammzPqjaZwrE8vN6oVMYpz3vAY/pob3nvqQgaY21U+K6njoF+fHHpHNEv1LyNVU6NEioU/CJf8KrviLFkq5mkqnqOEXGV1c22Vw9KwV7bS2P243d1/VK11G99EDtIEi9BztogN0iDqIBS74EfwMfoVvQhGaMD+TLgV1zF10zsJvfwD8G0Dj</latexit> ˆ d = arg min d2CL Gd udes H W Gd udes = GHW G + ⌘I 1 GHW udes [Koyama+ JAES 2023]
36. Weighted Pressure Matching Ø Comparison between Pressure Matching and Weighted Pressure Matching September 3, 2024 36 PM WPM (uniform) WPM (directional) Pressure Error [Koyama+ JAES 2023]
37. Spatial aliasing artifacts in sound field control Ø Owing to discrete placement of secondary sources, spatial aliasing artifacts are unavoidable – E.g., Synthesizing sound field by 12 loudspeakers at 800 Hz September 3, 2024 37 Desired Pressure Matching Pressure § Degradation of sound localization § Coloration of source signals
38. Idea for perceptual quality enhancement Ø Interaural level difference (ILD) is the dominant cue for horizontal sound localization above 1500 Hz, compared with interaural time difference (ITD) Ø Amplitude response should be accurately synthesized as much as possible, rather than phase response, to alleviate coloration effects September 3, 2024 38 Synthesizing amplitude (or magnitude) distribution leaving phase distribution arbitrary at high frequencies Applying amplitude matching for high frequencies Pressure Magnitude
39. Amplitude matching Ø Synthesizing desired amplitude at control points – By leaving phase arbitrary, number of parameters to be control can be reduced – First proposed for multizone sound field control for personal audio Ø Optimization problem of amplitude matching September 3, 2024 39 : Target region Secondary source [Koyama+ 2021, Abe+ 2023] Desired amplitude No closed-form solution, but majorization minimization (MM) algorithm or alternating direction method of multipliers (ADMM) can be applied Element-wise absolute value
40. Amplitude matching September 3, 2024 40 https://youtu.be/oYw7kmpZcY4 Full version:
41. Proposed method for perceptual quality enhancement Ø Combination of pressure and amplitude matching – is determined so that for low frequencies and for high frequencies – For example, can be defined as sigmoid function September 3, 2024 41 <latexit sha1_base64="7PBZJNy5EyF+eucvkOfjzch+WsM=">AAADWHicfZLtbtMwFIadFthWPtaOn/yxqJA6WKummhiaNKkwJBACMSS6TarbynFOO2uxE9kOoqS+DK6CK4KrwUlTYF9YivLqPc/x8fFxkERcm273p1ep3rp9Z219o3b33v0Hm/XG1rGOU8VgwOIoVqcB1RBxCQPDTQSniQIqgghOgvPDPH7yBZTmsfxs5gmMBJ1JPuWMGmdN6j+I4JIL/g0mGQlEFlrCJRHUnAVBdmjH7y1+11oGtvH+AW75bRKAoduYLHL7TWjb+T+146xIUw4FbS1ZTHrjHn6GC9zRK3yB23hxbcrib07kOghpWSMsN5vUm91Ot1j4qvBL0UTlOpo0vO8kjFkqQBoWUa2Hfjcxo4wqw1kEtkZSDQll53QGQyclFaBHWXGrFj9xToinsXKfNLhw/83IqNB6LgJH5l3oy7HcvC42TM30xSjjMkkNSLYsNE0jbGKcjwiHXAEz0dwJyhR3Z8XsjCrKjBtk7UKZQOwUN6in2rrIa3BtKvjgrI8JKGpi9TQjVM0E/Wpd2zOyk6v/gVyuQKduAt0mxZux2R91I1o+L4euVM0N0r88tqviuNfxn3d2P+02+6/Kka6jR+gxaiEf7aE+eouO0AAxr+HteX3vZeVXFVXXqhtLtOKVOQ/RhVXd+g1mbRVc</latexit> minimize d2CL J(d) := (1 )kGd udesk2 2 + k|Gd| |udes|k2 2 + kdk2 2 <latexit sha1_base64="A4dv1PYDkl/3NJ52frUgh4Qg2ME=">AAACsHicfZFNSyNBEIY7o66a1fVjj14Gg7CIhBkR9SjuHvayqGA0kAmhplMTW7t7hu4aMTvMb9iToD/Nf7M9MRG/Cxoe3nqru6orzqSwFAQPNW9qeubL7Nx8/evC4rel5ZXVM5vmhmOLpzI17RgsSqGxRYIktjODoGKJ5/HVzyp/fo3GilSf0jDDroKBFongQE5qRTES9JYbQTMYhf8WwjE02DiOeyu126if8lyhJi7B2k4YZNQtwJDgEst6lFvMgF/BADsONSi03WLUbelvOKXvJ6lxR5M/Up9XFKCsHarYORXQhX2dq8T3cp2ckv1uIXSWE2r++FCSS59Svxrd7wuDnOTQAXAjXK8+vwADnNwH1V88E6ut6n5jE1u6zC90Yxr846SjDA1QajaLCMxAwU3pxh5EWxV9ZhR6YnT0kdFdIpT4i2XxRB9ahZ5YJ1R3iwxfr+0tnG03w93mzslO4+BwvNI5tsbW2Q8Wsj12wH6zY9ZinAn2j92xe2/ba3s9Dx6tXm1c8529CO/yPyvr2Vk=</latexit> <latexit sha1_base64="v6ZpLa9I3hnLLs2O89d3S1l6XjM=">AAACsnicfZFNaxsxEIblzUdTN9899rLEFEIIZjeYNJdAaHPopTSBOA7xmjArzzrCknaRZkPcZX9ED7m0vyz/JlrHDvkeEDy88440o4kzKSwFwW3Nm5mdm/+w8LH+aXFpeWV1bf3Uprnh2OapTM1ZDBal0NgmQRLPMoOgYomdePijyneu0FiR6hMaZdhTMNAiERzISZ0oRoL98GK1ETSDcfgvIZxAg03i6GKtdhP1U54r1MQlWNsNg4x6BRgSXGJZj3KLGfAhDLDrUINC2yvG/Zb+V6f0/SQ17mjyx+rjigKUtSMVO6cCurTPc5X4Wq6bU7LXK4TOckLN7x9KculT6lfD+31hkJMcOQBuhOvV55dggJP7ovqTZ2K1Xd1vbGJLlzlEN6bBX076naEBSs1WEYEZKLgu3diDaLui94xCT42O3jK6S4QSf7AsHuhNq9BT65TqbpHh87W9hNOdZrjbbB23GgffJytdYF/YBttkIfvGDthPdsTajLMh+8v+sf9eyzv3wOP3Vq82qfnMnoQn7wB2ednb</latexit> = 1 <latexit sha1_base64="nFwv8TaL4YCj47fcATfYtfC3o0Y=">AAACsnicfZFNSyNBEIY74+6qcT/8OHoZNgjLImFGwrqXBVEPXkQFY2QzQWo6NbFJd8/QXbNsHOZHePCiv8x/Y09MxO+Choe33uqu6oozKSwFwW3Nm/nw8dPs3Hx94fOXr98Wl5ZPbJobjm2eytScxmBRCo1tEiTxNDMIKpbYiYc7Vb7zD40VqT6mUYY9BQMtEsGBnNSJYiT4E5wtNoJmMA7/JYQTaLBJHJ4t1a6ifspzhZq4BGu7YZBRrwBDgkss61FuMQM+hAF2HWpQaHvFuN/SX3NK309S444mf6w+rihAWTtSsXMqoHP7PFeJr+W6OSW/e4XQWU6o+f1DSS59Sv1qeL8vDHKSIwfAjXC9+vwcDHByX1R/8kys1qv7jU1s6TK76MY0uO+kgwwNUGp+FhGYgYL/pRt7EK1X9J5R6KnR0VtGd4lQ4gLL4oHetAo9tU6p7hYZPl/bSzjZaIa/mq2jVmNre7LSObbKvrMfLGSbbIvtsUPWZpwN2SW7Zjdey/vrgcfvrV5tUrPCnoQn7wB0Mdna</latexit> = 0 <latexit sha1_base64="A4dv1PYDkl/3NJ52frUgh4Qg2ME=">AAACsHicfZFNSyNBEIY7o66a1fVjj14Gg7CIhBkR9SjuHvayqGA0kAmhplMTW7t7hu4aMTvMb9iToD/Nf7M9MRG/Cxoe3nqru6orzqSwFAQPNW9qeubL7Nx8/evC4rel5ZXVM5vmhmOLpzI17RgsSqGxRYIktjODoGKJ5/HVzyp/fo3GilSf0jDDroKBFongQE5qRTES9JYbQTMYhf8WwjE02DiOeyu126if8lyhJi7B2k4YZNQtwJDgEst6lFvMgF/BADsONSi03WLUbelvOKXvJ6lxR5M/Up9XFKCsHarYORXQhX2dq8T3cp2ckv1uIXSWE2r++FCSS59Svxrd7wuDnOTQAXAjXK8+vwADnNwH1V88E6ut6n5jE1u6zC90Yxr846SjDA1QajaLCMxAwU3pxh5EWxV9ZhR6YnT0kdFdIpT4i2XxRB9ahZ5YJ1R3iwxfr+0tnG03w93mzslO4+BwvNI5tsbW2Q8Wsj12wH6zY9ZinAn2j92xe2/ba3s9Dx6tXm1c8529CO/yPyvr2Vk=</latexit> <latexit sha1_base64="ROtRu2ks9BH5ktUl1T0BgfcB3qQ=">AAADB3icfVHNbhMxEHaWv7L8peXIxSJCaqGNdqsKekGqgAMXRJGatiIOkdeZ3Vpd2yvbiwiWH4AH4Dm4IY7wGLwAV3gEvMkG0ZYykuVP33zjGX+TVSU3Nkm+d6ILFy9dvrJ0Nb52/cbNW93llX2jas1gwFSp9GFGDZRcwsByW8JhpYGKrISD7Phpkz94C9pwJffstIKRoIXkOWfUBmrcfU0ysHSVKAEFXcOPcUxyTZlLvUsfEEHtkRYO/Bu3MeeJ4YWg3m2Sivu2bGN+jd1Cv+f9mvfjbi/pJ7PAZ0Hagh5qY3e83PlIJorVAqRlJTVmmCaVHTmqLWcl+JjUBirKjmkBwwAlFWBGbmaCx/cCM8G50uFIi2fs3xWOCmOmIgvKZkxzOteQ/8oNa5tvjxyXVW1BsnmjvC6xVbhxFE+4BmbLaQCUaR5mxeyIBrNs8D0+0SYT6zOLTG58yDyD8E0NLwL1sgJNrdL3HaE6OPzOh28XZL1B/xNyuRAGdJ4wPMIFfw/e/UHnSrlcSBcoDotMT6/tLNjf7KcP+1uvtno7T9qVLqE76C5aRSl6hHbQc7SLBoihr+gH+ol+RR+iT9Hn6MtcGnXamtvoRETffgMYmvzf</latexit> (!) = 1 1 + e 2⇡ (! !T) Transition frequency Can still be solved by MM algorithm or ADMM Pressure matching Amplitude matching
42. Numerical experiments Ø Setting – 3D free field – Target region : Cuboid of 1.0 m x 1.0 m x 0.04 m – 32 loudspeakers on borders of squares of 2.0 m x 2.0 m at z=±0.1 m – 1152 control points regularly placed over every 0.04 m – Desired sound field: point source at (2.0 m, 0.0 m, 0.0 m) – Proposed method and pressure matching (PM) are compared September 3, 2024 42 <latexit sha1_base64="aJVm6ibCtbqRlu7KgruECMVr7Qw=">AAACvHicfZFNT+MwEIbdwAJbdvk87iWiQkJoVSUIASeEgAMXVFaiBamp0MSdBFPbiWwHUaL8B65w4m/xb3D6saJ8jWTp0TuvPTOeMOVMG897qThT0z9mZud+Vud//V5YXFpeaekkUxSbNOGJugxBI2cSm4YZjpepQhAhx4uwd1TmL25RaZbIc9NPsSMglixiFIyVWkFDYAxXSzWv7g3C/Qj+CGpkFGdXy5XnoJvQTKA0lIPWbd9LTScHZRjlWFSDTGMKtAcxti1KEKg7+aDdwl23SteNEmWPNO5AfXsjB6F1X4TWKcBc6/e5Uvws185MtNfJmUwzg5IOC0UZd03ilrO7XaaQGt63AFQx26tLr0EBNfaHJqqEYmKGvKyldKSLajU4RjuywlMrNVJUYBK1mQegYgF3hf2COPhb0ndGJsdGS18Z7SNMsHss8v/0pZXJsXVMdqf++w1+hNZW3d+pb//brh0cjrY7R/6QNbJBfLJLDsgJOSNNQskNeSCP5MnZd7pOzxFDq1MZ3VklE+HcvgLkeN8p</latexit> ⌦ <latexit sha1_base64="aJVm6ibCtbqRlu7KgruECMVr7Qw=">AAACvHicfZFNT+MwEIbdwAJbdvk87iWiQkJoVSUIASeEgAMXVFaiBamp0MSdBFPbiWwHUaL8B65w4m/xb3D6saJ8jWTp0TuvPTOeMOVMG897qThT0z9mZud+Vud//V5YXFpeaekkUxSbNOGJugxBI2cSm4YZjpepQhAhx4uwd1TmL25RaZbIc9NPsSMglixiFIyVWkFDYAxXSzWv7g3C/Qj+CGpkFGdXy5XnoJvQTKA0lIPWbd9LTScHZRjlWFSDTGMKtAcxti1KEKg7+aDdwl23SteNEmWPNO5AfXsjB6F1X4TWKcBc6/e5Uvws185MtNfJmUwzg5IOC0UZd03ilrO7XaaQGt63AFQx26tLr0EBNfaHJqqEYmKGvKyldKSLajU4RjuywlMrNVJUYBK1mQegYgF3hf2COPhb0ndGJsdGS18Z7SNMsHss8v/0pZXJsXVMdqf++w1+hNZW3d+pb//brh0cjrY7R/6QNbJBfLJLDsgJOSNNQskNeSCP5MnZd7pOzxFDq1MZ3VklE+HcvgLkeN8p</latexit> ⌦
43. Numerical experiments Ø Evaluation of ILD – Binaural signals in the synthesized sound field were calculated by using transfer functions from loudspeakers to a listener obtained by Mesh2HRTF [Ziegelwanger+ 2015] – Evaluation measure was normalized error of ILD: – Distribution of NE September 3, 2024 43 <latexit sha1_base64="EndMQejlYdBkvqIegxmgN2TX4IM=">AAAD5niclVLdatRAFJ40amv8a+ulN4OL0Eq7JFLa3hSKVqjgTwW3LWyWZTJ7sjs0MwkzE3Gdzit4pXjb5/BWX8K3cbLdSLe7FXsg8OU73/mZc05SZEzpMPztzfk3bt6aX7gd3Ll77/6DxaXlQ5WXkkKL5lkujxOiIGMCWprpDI4LCYQnGRwlJy8q/9FHkIrl4oMeFtDhpC9YyijRjuoueesxJ3oguXn70q7ECTfSdk3N7Vu7indwEKeSUBOrkjtfMWATCotP679Xr/cuRKuhsDNzrk3lWF2fnULLEv47xym21+7xegVsd7ERNsOR4WkQjUEDje3ADfgs7uW05CA0zYhS7SgsdMcQqRnNwAZxqaAg9IT0oe2gIBxUx4wWa/ETx/Rwmkv3CY1H7MUIQ7hSQ544ZdWmuuyryFm+dqnT7Y5hoig1CHpeKC0zrHNcXQnuMQlUZ0MHCJXM9YrpgLgj0O6WJqokfOINZjQulSobBPEeuCdLeOOodwVIonP51MRE9jn5ZN0I+vFahf4lZKIWOnSV0CVhnH0Ga/6iK6VM1NIauZ1Glzc4DQ6fNaPN5sb7jcbu8/F2F9Aj9BitoAhtoV20jw5QC1Hvq/fD++n98gf+F/+b//1cOueNYx6iCfPP/gCusFhL</latexit> NE(r H ) = P H |ILD syn (r H , H ) ILD true (r H , H )| P H |ILD true (r H , H )| Position and direction of listener’s head PM Proposed
44. Numerical experiments Ø Evaluation of amplitude response – At origin September 3, 2024 44
45. Listening experiments Ø Evaluation by MUSHRA – Desired sound field: point source at (2.0 m, 0.5 m, 0.0 m) – Reverberation time (T60 ): 0.19 s – 14 male subjects in 20-30s – Listening at center of target region – Test signals: • Reference: Source signal from reference loudspeaker • C1/Hidden anchor: lowpass-filtered source signal up to 3.5 kHz • C2/PM: Synthesized sound by PM • C3/Proposed: Synthesized sound by Proposed • C4/Hidden reference: Same as reference September 3, 2024 45
46. Listening experiments Ø Results of two source signals (Vocals/Instrumental) September 3, 2024 46 Vocals Instrumental C1/Hidden anchor C2/PM C3/Proposed C4/Hidden reference Synthesized sound by Proposed is perceptually close to reference sound compared to PM
47. Application to Spatial Active Noise Control Ø Environmental noise is still unsolved problem Ø Active noise control (ANC) is aimed to cancel noise by loudspeaker signals, but its effect is limited to local region Ø ANC in 3D space based on sound field analysis/synthesis September 3, 2024 47 Noise suppression by loudspeaker signals Quiet zone
48. Application to Spatial Active Noise Control Ø Cost function of regional noise power is estimated by kernel interpolation of sound field Ø Adaptive filtering algorithm based on kernel interpolation is also derived September 3, 2024 48 ANC in 3D space based on sound field interpolation Ø Conventional cost function Ø Proposed cost function <latexit sha1_base64="Pnlxe5THpW3gUCE+pMJvaUIbsmQ=">AAACFXicbZDLSsNAFIYnXmu9RV2KMFgUVyUpRd0IRTcuXFSwF2himUwn7dCZJMxMhBKz8iF8Bre6diduXbv0TZy0WdjWHwY+/nMO58zvRYxKZVnfxsLi0vLKamGtuL6xubVt7uw2ZRgLTBo4ZKFoe0gSRgPSUFQx0o4EQdxjpOUNr7J664EIScPgTo0i4nLUD6hPMVLa6poHDkdqgBFLblJ4fAGdR+h4PCGppvtK1yxZZWssOA92DiWQq941f5xeiGNOAoUZkrJjW5FyEyQUxYykRSeWJEJ4iPqkozFAnEg3GX8jhUfa6UE/FPoFCo7dvxMJ4lKOuKc7s6PlbC0z/6t1YuWfuwkNoliRAE8W+TGDKoRZJrBHBcGKjTQgLKi+FeIBEggrndzUFo+nOhN7NoF5aFbK9mm5elst1S7zdApgHxyCE2CDM1AD16AOGgCDJ/ACXsGb8Wy8Gx/G56R1wchn9sCUjK9f5U+ekQ==</latexit> L = kek2 : Power of error mics : Regional noise power <latexit sha1_base64="pm/hhsXKsYUZ/lWoZalEg8K5t2U=">AAACNHicbVDLSgMxFM34rPVVdekmWJS6KTOlqAuFohsXghXsAzq1ZNK0DU0yQ5IRynR+xo/wG9zqTnAj4tZvMNN2YVsPBA7nnMu9OV7AqNK2/W4tLC4tr6ym1tLrG5tb25md3aryQ4lJBfvMl3UPKcKoIBVNNSP1QBLEPUZqXv8q8WuPRCrqi3s9CEiTo66gHYqRNlIrc+5ypHsYsegmhkcX0KVCtyL3lpMuiuEwzLkej2R8PHwowFFU8qgdj0XYymTtvD0CnCfOhGTBBOVW5tNt+zjkRGjMkFINxw50M0JSU8xInHZDRQKE+6hLGoYKxIlqRqNfxvDQKG3Y8aV5QsOR+nciQlypAfdMMjlUzXqJ+J/XCHXnrBlREYSaCDxe1AkZ1D5MKoNtKgnWbGAIwpKaWyHuIYmwNsVObfF4bDpxZhuYJ9VC3jnJF++K2dLlpJ0U2AcHIAcccApK4BqUQQVg8ARewCt4s56tD+vL+h5HF6zJzB6YgvXzC9DNrBs=</latexit> L = Z ⌦ |u(r)|2dr [Ito+ IEEE ICASSP 2019 (Best Student Paper Award), Koyama+ IEEE/ACM TASLP 2021]
49. Application to Spatial Active Noise Control Ø Band-limited noise (500-800Hz), T60 : 240ms September 3, 2024 49 Proposed: -10.5 dB MPC: -6.2 dB (dB) Regional noise reduction is achieved by the proposed method
50. Conclusion Ø Physics-informed sound field estimation and control qSound field estimation: • Kernel interpolation of sound field with constraint of Helmholtz equation • Kernel function adapted to acoustic environment with the aid of neural networks qSound field control: • Combination of pressure and amplitude matching for perceptual quality enhancement • Spatial active noise control based on kernel interpolation September 3, 2024 50 Thank you for your attention!