Unfortunately this is a huge misconception regarding asteroids. First of all, what is the asteroid belt? The asteroid belt is one location of asteroids in our Solar System. Notice I said "one location". There are other locations of asteroids in the Solar System, most notably in Jupiter's orbit. The asteroid belt itself is a region of asteroids between Mars and Jupiter that orbit the Sun. If you look at pictures in textbooks you'll see something like the following.
|The Asteroid Belt is marked by the red arrow.|
Looking at the picture, it seems as if the entire region between Mars and Jupiter is filled with asteroids. In fact, the opposite is true. The distance between two asteroids in the asteroid belt is on average a few million miles. That's a huge distance! In fact, you can randomly travel through the asteroid belt for years and your chances of hitting an asteroid are slim to none!
The problem with textbook pictures like this is that the size of a single pixel in the image is many times bigger than the actual asteroid scaled. If you were to scale the asteroid down to the scale of this image, even the largest asteroid would be invisible! But invisible pixels don't illustrate the location of asteroids very well, so "tiny" dots are placed on the image.
To conclude, the asteroid belt, although containing more asteroids than other regions of the Solar System, is still extremely empty. So have no fear the next time you find yourself traveling to Jupiter!