They already do something similar to this, its called Endurals, they race against time on roads, two-tracks, trails, in the woods, in fields, etc..(motorcycles) For cars, Yes, in a sense, it can be a smooth, well kept dirt road "OR" they're nothing more than a path or a two-track, a rough, un-kept access lane, it also depends on the promoter or sanctioning body putting it on and the type of Rally it is.
Chat with our AI personalities