If feasible, I always take the broom and try to clear away debris where I know people ride.
I still remember in my old house all of the bikes that would stop and wave or say thank you when they saw me out in front of my house sweeping up all of the gravel that had accumulated in the intersection after the winter.
I do need to take a ride up in my cage to the curves near my house and hit them with the broom. Been wanting to do that some evening for a couple weeks. Thanks for the inspiration
Also, don't be afraid to call your county traffic dept to notify them of the hazard. Most will make it a priority to get out there to fix it.
Just be careful out there when sweeping, would hate for some careless cager to run you over while doing a good deed.
If we all took an hour to hit the corners of our favorite local rides a couple times a year, think of how nice they would all be for everyone.