Claudia Totzeck (Bergische Universität Wuppertal, Germany) Adjoint-based Optimal Control with Wasserstein-2 Metric
WORKSHOP ON OPTIMAL TRANSPORT
FROM THEORY TO APPLICATIONS
INTERFACING DYNAMICAL SYSTEMS, OPTIMIZATION, AND MACHINE LEARNING
Venue: Humboldt University of Berlin, Dorotheenstraße 24
Adjoints in W2 setting joint work with: M. Burger, R. Pinnau, O. Tse 3) Numerical examples: sheep & drones, fingerprints joint work with: M. Burger, L.M. Kreusser, R. Pinnau, O. Tse, A. Roth C. Totzeck (University of Wuppertal) OC with W2-metric 2/29
control functional J(y, u) = T 0 1 2 |y(t) − ydes (t)|2 + λ 2 |u(t)|2dt subject to the heat equation ∂t y = ∆y + u, y(0, ·) = 0 in Ω, y|∂Ω = 0 on Γ and set of admissible controls U = u ∈ L2(Ω × (0, T)) . Well-known facts [Tr¨ oltzsch, Hinze/Pinnau/Ulbrich/Ulbrich] well-posedness of the state equation ⇒ control-to-state operator: S : U → W (0, T) := y ∈ L2(0, T; H1(Ω): ∂t y ∈ L2(0, T; H−1(Ω)) S(u) = y existence and uniqueness of optimal control u C. Totzeck (University of Wuppertal) OC with W2-metric 3/29
= S(u), i.e. S (u) = −dy e(S(u), u)−1eu (S(u), u) Reduced cost functional: G(u) := J(S(u), u) = T 0 1 2 |S(y)(t) − ydes (t)|2 + λ 2 |u(t)|dt Note: Unconstrained optimization (constraint implicit in S) Adjoint-based gradient descent algorithm for G(u) directional derivative: for all feasible directions h: 0 ! = G (u)[h] = dy J(S(u), u), S (u)[h] Y + du J(S(u), u), h U = S (u) ∗ dy J(S(u), u) + du J(S(u), u), h U adjoint equation: ey (S(u), u)∗p = −Jy (S(u), u) Riesz representation of the gradient: ∇G(u) = Ju (S(u), u) − eu (S(u), u)∗p C. Totzeck (University of Wuppertal) OC with W2-metric 4/29
u0 = ˆ u} We seek to find a minimizer of min C([0,T],P2(Rd ))×Uad J(ρ, u) = J1 (ρ) + J2 (u) subject to ∂t ρt + ∇x · (v(ρt , ut )ρt ) = 0. Assumptions on velocity field v : v : P2 (Rd ) × RdM → Liploc (Rd ) such that for all (ρ, u) v(ρ, u)(x) − v(ρ, u)(y), x − y ≤ Cl |x − y|2 x, y ∈ Rd for Cl > 0 independent of (ρ, u), for any two (ρ, u), (ρ , u ) exists Cv > 0 indep. of (ρ, u), (ρ , u ): v(ρ, u) − v(ρ , u ) sup ≤ Cv W2 (ρ, ρ ) + u − u 2 . C. Totzeck (University of Wuppertal) OC with W2-metric 7/29
equation [Dobrushin, Golse, ...] For given u ∈ Uad , v(ρ, u) as above and ρ(0, ·) = ρ0 ∈ P2 (Rd ) ∂t ρ(t, x) + ∇x · (v(ρt , ut )(x)ρ(t, x)) = 0 in [0, T] × Ω (1) admits a unique solution ρin ∈ P(Ω) for any T > 0. We define ρ = S(u). Stability estimate for the state equation For weak solutions ρ, ¯ ρ of (1) with initial data ρ0 , ¯ ρ0 and controls u, ¯ u, respectively, one can show that there exists positive constants a, b such that W 2 2 (ρt , ¯ ρt ) ≤ W 2 2 (ρ0 , ¯ ρ0 ) + b u − ¯ u 2 L2(0,T;RdM ) exp(at) (2) for all t ∈ [0, T]. In particular, this shows the weak-continuity of the solution operator. Example: v(ρt , ut ) = −K1 ∗ ρt + K2 (·, ut ) C. Totzeck (University of Wuppertal) OC with W2-metric 8/29
min C([0,T],P2(Rd ))×Uad J(ρ, u) = J1 (ρ) + J2 (u) subject to ∂t ρt + ∇x · (v(ρt , ut )ρt ) = 0. Assumptions on cost functional J : J1 (ρ) cylindrical, i.e. J1 (ρ) = j( g1 , ρ , . . . , gL , ρ ) with j ∈ C1(RL), g ∈ C1(Rd ), = 1, . . . , L such that g , ρ = Rd g dρ < ∞ and |∇g | ≤ Cg (1 + |x|) for all x ∈ Rd , = 1, . . . , L for some Cg > 0, J2 ∈ C1, w.l.s. and coercive. Example J1 consisting of moments of ρ (variance + tracking of expectation) J2 kinetic energy C. Totzeck (University of Wuppertal) OC with W2-metric 9/29
W2 is narrow lower semicontinuous. Let un be minimizing sequence, i.e., G(u∗) := J(S(u∗), u∗) = lim inf n J(S(un ), un ) = inf u {J1 (S(u)) + J2 (u)}. Due to the coercivity of J2 the minimizing sequence is bounded. We can extract a weakly convergent subsequence un u∗. By (2) S is weakly continuous, hence ρ(un ) ρ(u∗). And we estimate W2 (ρ(u∗), ρdes )2 + J2 (u∗) ≤ lim inf n→∞ W2 (ρ(un ), ρdes )2 + W2 (ρ(un ), ρu )2 + J2 (un ) = W2 (S(u∗), ρdes )2 + J2 (u∗) = G(u∗) This proves the existence of a minimizer. C. Totzeck (University of Wuppertal) OC with W2-metric 10/29
Hamiltonian: H(ν, u) = 1 2 (r − r ) · K(x − x , ut )dνt dνt − j1 (ρt ) − j2 (ut ) PMP (short) If u is an optimal control and ρ the corresponding solution, then there exist ν such that u = arg max u∗∈Uad H(ν∗, u∗), ∂t ν = −∇(x,r) · S∇νH(ν, u) ν , S = 0 Id −Id 0 ν0 (E × Rd ) =ρ(0), νT (Rd × E) = δ0 Note: evolution of ν is forward-backward coupled! very difficult for numerical approaches C. Totzeck (University of Wuppertal) OC with W2-metric 11/29
the Hilbert case: G (u)[h] ! = 0 for all feasible directions h adjoint interfaces variation of state and cost w.r.t. u and y: S (u) = −ey (S(u), u)−1eu (S(u), u) ⇒ ey (S(u), u)∗p = −Jy (S(u), u) Ju (S(u), u) − eu (S(u), u)∗p = 0 Plan to establish adjoint in W2 sense 1) compute G (u)[hu ] 2) establish relationship of variation S(u) and S(u + δh) in particular, dy e(S(u), u)[hy ] and du e(S(u), u)[hu ] 3) compute the adjoint equation 4) assemble the first-order optimality condition C. Totzeck (University of Wuppertal) OC with W2-metric 12/29
the Hilbert case: G (u)[h] ! = 0 for all feasible directions h adjoint interfaces variation of state and cost w.r.t. u and y: S (u) = −ey (S(u), u)−1eu (S(u), u) ⇒ ey (S(u), u)∗p = −Jy (S(u), u) Ju (S(u), u) − eu (S(u), u)∗p = 0 Plan to establish adjoint in W2 sense 1) compute G (u)[hu ] 2) establish relationship of variation S(u) and S(u + δh) in particular, dy e(S(u), u)[hy ] and du e(S(u), u)[hu ] 3) compute the adjoint equation 4) assemble the first-order optimality condition C. Totzeck (University of Wuppertal) OC with W2-metric 12/29
given h such that u + δh ∈ Uad for all δ ∈ [0, 1] define ρδ = S(u + δh) and ρ = S(u). By our control-to-state stability result it holds W 2 2 (ρδ t , ρt ) ≤ δ √ beaT/2 h L2(0,T;RdM ) for appropriate a, b > 0. ⇒ δ → ρδ t ∈ P2 (Rd ) absolutely continuous ⇒ existence of vector field ψ ∈ L2(ρt , Rd ) for each t ∈ [0, T] satisfying lim δ→0 W2 (ρδ t , (id + δψ)# ρt ) δ = 0 Moreover, with explicit coupling πt (x) = (id + δψt , id)# ρt get W 2 2 ((id + δρt )# ρt , ρt ) ≤ |x + δψ(x) − x|2dρt (x) = δ2 |ψ(x)|2dρt In particular, lim sup δ→0 W 2 2 (ρδ t , ρt ) δ = lim sup δ→0 W 2 2 ((id + δψt )# ρt , ρt ) δ = ψ L2(ρt ,Rd ) Next: Find evolution equation for ψ! C. Totzeck (University of Wuppertal) OC with W2-metric 14/29
given h such that u + δh ∈ Uad for all δ ∈ [0, 1] define ρδ = S(u + δh) and ρ = S(u). By our control-to-state stability result it holds W 2 2 (ρδ t , ρt ) ≤ δ √ beaT/2 h L2(0,T;RdM ) for appropriate a, b > 0. ⇒ δ → ρδ t ∈ P2 (Rd ) absolutely continuous ⇒ existence of vector field ψ ∈ L2(ρt , Rd ) for each t ∈ [0, T] satisfying lim δ→0 W2 (ρδ t , (id + δψ)# ρt ) δ = 0 Moreover, with explicit coupling πt (x) = (id + δψt , id)# ρt get W 2 2 ((id + δρt )# ρt , ρt ) ≤ |x + δψ(x) − x|2dρt (x) = δ2 |ψ(x)|2dρt In particular, lim sup δ→0 W 2 2 (ρδ t , ρt ) δ = lim sup δ→0 W 2 2 ((id + δψt )# ρt , ρt ) δ = ψ L2(ρt ,Rd ) Next: Find evolution equation for ψ! C. Totzeck (University of Wuppertal) OC with W2-metric 14/29
given h such that u + δh ∈ Uad for all δ ∈ [0, 1] define ρδ = S(u + δh) and ρ = S(u). By our control-to-state stability result it holds W 2 2 (ρδ t , ρt ) ≤ δ √ beaT/2 h L2(0,T;RdM ) for appropriate a, b > 0. ⇒ δ → ρδ t ∈ P2 (Rd ) absolutely continuous ⇒ existence of vector field ψ ∈ L2(ρt , Rd ) for each t ∈ [0, T] satisfying lim δ→0 W2 (ρδ t , (id + δψ)# ρt ) δ = 0 Moreover, with explicit coupling πt (x) = (id + δψt , id)# ρt get W 2 2 ((id + δρt )# ρt , ρt ) ≤ |x + δψ(x) − x|2dρt (x) = δ2 |ψ(x)|2dρt In particular, lim sup δ→0 W 2 2 (ρδ t , ρt ) δ = lim sup δ→0 W 2 2 ((id + δψt )# ρt , ρt ) δ = ψ L2(ρt ,Rd ) Next: Find evolution equation for ψ! C. Totzeck (University of Wuppertal) OC with W2-metric 14/29
)# ρt . Lemma 1 Let (ρ, u) be an admissible pair, h ∈ C∞ c (0, T; RdM ) and uδ = u + δh such that (i) uδ ∈ Uad and, (ii) there existis ρδ ∈ C([0, T], RdM ) satisfying S(uδ) = ρδ for 0 < δ 1 sufficiently small. If ψ ∈ C1 b ((0, T) × Rd ) with ψ0 ≡ 0 satisfies ∂t ψ + Dψv(ρt , ut ) = K(ρt , ut )[ψt , ht ] for ρt -almost every x ∈ Rd for a bounded Borel map (t, x) → K(ρt , ut )[ψt , ht ](x) satisfying lim δ→0 T 0 v(νδ t , uδ t ) ◦ (id + δψ)(x) − v(ρt , ut )(x) δ − K(ρt , ut )[ψt , ht ](x) 2 dρt (x)dt = 0, then with this ψ is holds lim δ→0 W2 (ρδ t , (id + δψ)# ρt ) δ = 0. C. Totzeck (University of Wuppertal) OC with W2-metric 15/29
δψt )# ρt Step 1: Show t → νδ t is absolutely continuous curve satisfying ∂t νδ t + ∇ · (bδ t νδ t ) = 0 in sense of distributions where bt := δK(ρt , ut )[ψt , ht ] + v(ρt , ut ) ◦ (id + δψt )−1. This justifies Step 2: consider the temporal derivative t → W 2 2 (ρδ t , νδ t ) and estimate with the help of the assumptions and our stability estimate d dt W 2 2 (ρδ t , νδ t ) ≤ CW 2 2 (ρδ t , νδ t ) + δ2eδ t , with eδ t := v(νδ t ,uδ t )◦(id+δψ)(y)−v(ρt ,ut )(y) δ − K(ρt , ut )[ψt , ht ](y) 2 dρt . Since W 2 2 (ρδ 0 , νδ 0 ) = 0 our assumptions together with an Gronwall-type argument yields sup t∈[0,T] W 2 2 (ρδ t , νδ t ) δ2 ≤ eCT T 0 eδ s ds −→ 0. C. Totzeck (University of Wuppertal) OC with W2-metric 16/29
term eδ t can be computed explicity: Let v(ρt , ut ) = −(K1 ∗ ρt )(x) − M =1 K2 (· − ut ), then K(ρt , ut )[ψt , ht ] = Dv(ρt , ut )[ψt ] + (DK1 )(· − y)ψ(y)dρt (y) + M =1 (DK2 )(· − ut )h . Theorem 2 The our assumptions hold. For the explicit velocity field above v(ρt , ut ): P2 (Rd ) × RdM → Liploc (Rd ) there exists ψ ∈ C1 b ((0, T) × Rd ) with ψ0 ≡ 0 satisfying ∂t ψ + Dψv(ρt , ut ) = K(ρt , ut )[ψt , ht ] for ρt -almost every x ∈ Rd . Idea of the proof: fixed-point argument. C. Totzeck (University of Wuppertal) OC with W2-metric 17/29
(vector-valued measures) We test the equation for ψ to obtain T 0 ∂t ψ + Dψv(ρt , ut ) − K(ρt , ut )[ψt , ht ] · dmt dt = 0 Using ψ0 ≡ 0 and integration by parts yields T 0 ∂t mt + ∇· v(ρt , ut ) ⊗ mt + K1,∗(ρt , ut )[mt ], ψt dt = ψT · dmT − T 0 K2,∗(ut )[ht ] · dmt dt, where K1,∗(ρt , ut )[mt ] = ∇v(ρt , ut )mt + ρt (∇K1)(y − ·)dmt (y). Now, define adjoint equation following the idea of the Hilbert case! C. Totzeck (University of Wuppertal) OC with W2-metric 18/29
− K1(y − x) · ∇qt (y)ft (y)dy+v(ft , ut ) · ∇qt = L i=1 ∂i j( g1 , ft dx , . . . , gK , ft dx )gi scalar no mass conservation used for the numerical results relation to W2 adjoint via the ansatz: mt = ρt ∇gt Particle approximation: state equation via: empirical measue ρN t = 1 N N i=1 δ xi t adjoint equation via mN t = 1 N N i=1 ξi t δ xi t , where d dt ξi t = 1 N N j=1 ∇K1(xi t − xj t )(ξj t − ξi t ) + M =1 ∇K2(xi t − ut )ξi t + M i=1 (∂i j)( g1 , ρN t , . . . , gk , ρN t )∇gi Relation to PMP: measure ν driven by both, state & adjoint characteristics C. Totzeck (University of Wuppertal) OC with W2-metric 20/29
− K1(y − x) · ∇qt (y)ft (y)dy+v(ft , ut ) · ∇qt = L i=1 ∂i j( g1 , ft dx , . . . , gK , ft dx )gi scalar no mass conservation used for the numerical results relation to W2 adjoint via the ansatz: mt = ρt ∇gt Particle approximation: state equation via: empirical measue ρN t = 1 N N i=1 δ xi t adjoint equation via mN t = 1 N N i=1 ξi t δ xi t , where d dt ξi t = 1 N N j=1 ∇K1(xi t − xj t )(ξj t − ξi t ) + M =1 ∇K2(xi t − ut )ξi t + M i=1 (∂i j)( g1 , ρN t , . . . , gk , ρN t )∇gi Relation to PMP: measure ν driven by both, state & adjoint characteristics C. Totzeck (University of Wuppertal) OC with W2-metric 20/29
[Dobrushin, Braun-Hepp, Neunzert, Golse, ...] W2 (ρN t , ρt ) ≤ CW2 (ρN 0 , ρ0 ) In particular, W2 (ρN 0 , ρ0 ) = O(1/ √ N) as N → ∞ if Xi 0 ∼ ρ0 iid. Q: Can we translate this to the convergence of controls? Theorem Let (¯ ρ, ¯ u) and (xN, uN) be the optimal pairs Xi 0 ∼ ρ0 iid for all i = 1, . . . , N. Then, there exists a constant c0 > 0 depending only on T and v such that for λ > c0 in J2 (convexity condition), it holds uN − ¯ u 2 ∞ ≤ c0 λ − c0 W 2 2 (ρN 0 , ρ0 ). Idea of the proof: stability estimate for the adjoint equation control sets for micro and macro coincide play with the optimality conditions C. Totzeck (University of Wuppertal) OC with W2-metric 21/29
[Dobrushin, Braun-Hepp, Neunzert, Golse, ...] W2 (ρN t , ρt ) ≤ CW2 (ρN 0 , ρ0 ) In particular, W2 (ρN 0 , ρ0 ) = O(1/ √ N) as N → ∞ if Xi 0 ∼ ρ0 iid. Q: Can we translate this to the convergence of controls? Theorem Let (¯ ρ, ¯ u) and (xN, uN) be the optimal pairs Xi 0 ∼ ρ0 iid for all i = 1, . . . , N. Then, there exists a constant c0 > 0 depending only on T and v such that for λ > c0 in J2 (convexity condition), it holds uN − ¯ u 2 ∞ ≤ c0 λ − c0 W 2 2 (ρN 0 , ρ0 ). Idea of the proof: stability estimate for the adjoint equation control sets for micro and macro coincide play with the optimality conditions C. Totzeck (University of Wuppertal) OC with W2-metric 21/29
xi = vi , i = 1, . . . , N, m = 1, . . . , M d dt vi = − 1 N N j=1 KS (xj − xi ) − M m=1 KD (dm , xi ), d dt dm = um , x(0) = x0 , v(0) = v0 , d(0) = d0 . attraction comfort repulsion cost functional: J(y, u) = σ1 4 (V(y) − V0 )2 + σ2 2 (E(y) − Edes )2 + λ 2 u 2 V0 given reference Variance, Edes the destination. C. Totzeck (University of Wuppertal) OC with W2-metric 23/29
xi = vi , i = 1, . . . , N, m = 1, . . . , M d dt vi = − 1 N N j=1 KS (xj − xi ) − M m=1 KD (dm , xi ), d dt dm = um , x(0) = x0 , v(0) = v0 , d(0) = d0 . attraction comfort repulsion cost functional: J(y, u) = σ1 4 (V(y) − V0 )2 + σ2 2 (E(y) − Edes )2 + λ 2 u 2 V0 given reference Variance, Edes the destination. C. Totzeck (University of Wuppertal) OC with W2-metric 23/29
measure ρN (0, x) = 1 N N i=1 δ(x − xi ), law(xi ) = ρ0 . First-order optimality system on particle level dxi dt = 1 N N j=1 ¯ F(xi − xj , u), i = 1, . . . , N, d dt ξi = 1 N N j=1 ∇x ¯ F(xi − xj , u)ξi − 1 N N j=1 ∇x ¯ F(xi − xj , u)ξj , dJ2 (uN ) = 1 N2 N i=1 N j=1 T 0 ∇u ¯ F(xi (t) − xj (t), uN )ξi (t)dt x(0) = x0 , ¯ ξ(T) = tρdes ρT (xi (T)) − xi (T). Note: terminal condition computed with POT [Flamary et al.] C. Totzeck (University of Wuppertal) OC with W2-metric 26/29
control and optimality conditions in the space of probability measures M. Burger, R. Pinnau, CT, O. Tse SICON 59 (2), pp. 977-1006, 2021. Mean-field optimal control for biological pattern formation M. Burger, L. M. Kreusser, CT ESAIM COCV 27 (40), 2021. Other references: Ambrosio, Gigli, Savar´ e (green book) Optimal control PDE, Tr¨ oltzsch Optimal control PDE, Hinze-Pinnau-Ulbrich-Ulbrich POT github repository: https://pythonot.github.io/ [email protected] C. Totzeck (University of Wuppertal) OC with W2-metric 29/29