Holding with a loose grip will create more spin which will increase speed. You won't "feel" like you're throwing hard because the grip is loose. But you don't want it so loose that you're throwing it to the backstop.
A tighter grip will usually slow down the ball. "Take a little off" means to throw it a slower speed.
One of the common mistakes pitchers make is they think a tighter grip will give more speed because you can feel "it". So you think it's fast. But speed on a fastball is a combination of how much force you put on the ball and how much spin.
You can experiment with that works for you and what feels comfortable.