Fifteen or twenty miles per hour on a surface like this is perfectly possible - more for an experienced rider. I wouldn't try it though till you have found your confidence with yourself and with the bike. Stay close to your comfort zone at first. Going too slowly though will make you fearful of the situation. Find what you are comfortable with and open the bike up just a tiny bit more.
Riding rough surfaces is all about learning to trust the bike. Only then will you stop thinking that you have to do something about every tiny little slither. The rake and trail of the bike are designed to make it self-correcting - so long as you don't make sudden inputs, moving bikes like to stay upright and to some extent will correct themselves. You have to get used to that idea.
Having said that, you also have to know the bike's limits - you still have to watch out for leaning or turning on loose gravel, for example.
But I'm running on ahead here and talking about off-road riding. The kind of gravel you usually find on roads is best avoided if you can. If you can't - treat it with caution and follow the advice everyone has been giving.
(We're metric in the UK for everything except distances. We do metres for everything except roads and the clocks in our vehicles still do mph.
)
Sometimes it pays to be a scruffy oik. My cleaning and polishing routine generally lasts for six months after getting a new bike. After that, the bike is for riding! 
