Hfrxdjt

I am trying to minimize the following function:

$$f(theta_1, dots, theta_n) = frac1ssum_j =1^sleft(sum_i=1^n sin(t_j + theta_i)right)^2$$

with respect to $(theta_1, dots, theta_n)$. Here $t_j$ are regularly spaced points in the interval $[0, 2pi)$.

So here is the Python (PyTorch) code for that. The optimization does not seem to be computed correctly (the gradients seem to only advance along the line $theta_1 = cdots = theta_n$, which of of course incorrect).

import numpy as np
import torch

def phaseOptimize(n, s = 48000, nsteps = 1000):
 learning_rate = 1e-3

 theta = torch.zeros([n, 1], requires_grad=True)
 l = torch.linspace(0, 2 * np.pi, s)
 t = torch.stack([l] * n)
 T = t + theta

 for jj in range(nsteps):
 loss = T.sin().sum(0).pow(2).sum() / s
 loss.backward()
 theta.data -= learning_rate * theta.grad.data

 print('Optimal theta: nn', theta.data)
 print('nnMaximum value:', T.sin().sum(0).abs().max().item())

Below is a sample output.

phaseOptimize(5, nsteps=100)


Optimal theta: 

 tensor([[1.2812e-07],
 [1.2812e-07],
 [1.2812e-07],
 [1.2812e-07],
 [1.2812e-07]], requires_grad=True)


Maximum value: 5.0

I am assuming this has something to do with broadcasting in

T = t + theta

and/or the way I am computing the loss function.

One way to verify that optimization is incorrect, is to simply evaluate the loss function at random values for the array $theta_1, dots, theta_n$, say uniformly distributed in $[0, 2pi]$. The maximum value in this case is almost always much lower than the maximum value reported by phaseOptimize(). Much easier in fact is to consider the case with $n = 2$, and simply evaluate at $theta_1 = 0$ and $theta_2 = pi$. In that case we get:

phaseOptimize(2, nsteps=100)

Optimal theta: 

 tensor([[2.8599e-08],
 [2.8599e-08]])


Maximum value: 2.0

On the other hand,

theta = torch.FloatTensor([[0], [np.pi]])
l = torch.linspace(0, 2 * np.pi, 48000)
t = torch.stack([l] * 2)
T = t + theta

T.sin().sum(0).abs().max().item()

produces

3.2782554626464844e-07