The shelf is doing work here as well. Pigeons get extremely protective of one specific spot, and if your attacks are happening in one place rather than everywhere, that place is the problem rather than you.
Take the nest-like quality out of it if he is not meant to be breeding: no nest material, no dark corner he can defend, and stay off the shelf unless you have a reason. Mine owns the top of a wardrobe and I have simply ceded it.