Well, I think you've identified one of those grey areas with your point...but in the context of a pitched ball the difference between correct usage and incorrect usage may not really matter all that much (except, perhaps, to physicists).
The ball's flight path 'almost' doesn't change on its way to HP...although, in reality, there is no such thing as a truly straight pitch in any gravitational field. Curves curve, of course, but so do fastballs and every other type of pitch. So, at some level of concern, which will always be directly proportional to a given pitch's amount of deviation from a perfect linear trajectory, average speed and average velocity are never truly interchangeable measures of a real pitched baseball.
Personally, I think 'velocity' is used more because it sounds more cool in some contexts. I.e., "muzzle velocity" is similar to the concept of "instantaneous velocity"....what the velocity of the ball is conceived to be at a single instant in time--this is almost what a radar gun tries to measure, and it is usually a high number because you can take radar readings very near the release point just as the ball leaves the muzzle of the pitcher's gun. And everybody wants to see high numbers, of course, so it makes sense that measurements become associated with muzzle velocity. So there's a lot of macho gun-related metaphor there, and it just sounds cool.
Lets face it, if you were a paleontologist would you rather study 'velociraptors' or 'speedy dinosaurs'?
On the other hand, 'raw speed' sounds much better than 'raw velocity'. Go figure.
Geez, Orangepeel, between you and me I hope no one else is actually reading this thread...