I think the relevant killer app is not going to be Grover's algorithm, but quantum annealing.

Minor comment here: quantum annealing can be thought of basically, as the equivalent of quantum walks, which are an extension of Grover's algorithm. I would be very surprised if there was any difference between them.

Other than that, I agree with your comment but also wanted to notice this article: https://journals.aps.org/prxquantum/abstract/10.1103/PRXQuantum.2.010103 which poses severe difficulty to the usefulness of any Grover-like advantage.