I just discovered the Buy Chariots skill isn't very useful either. Having two horses is no faster than riding one, and, you can't even attack while riding one, only shoot arrows.
So its kinda another one that just looks cool.
From what I've seen, the following seems to be true;
Horses have a high speed cap, but a pretty slow momentum buildup. Horses go fast on stone and grass but can't run full speed on sand.
Camels have a medium-high speed cap and a reasonable momentum buildup. Camels go fast on sand but can't run full speed on stone or grass.
Chariots have about the same speed cap as camels. Chariots seem to hit their speed cap almost instantly. Chariots can also drift around corners and seem to run the same speed on any somewhat flat terrain.
Use horses when travelling long distance, camels when travelling through big pieces of desert and chariots when travelling in a city.