I wouldn’t even call it pushing off with your fingertips. I would call it following the ball with your fingertips, not that either term is necessarily more correct, but pushing off may lead some to flick a little too hard and create top spin.
The reason they say to “push” the ball in those tutorials is usually because that’s their way of saying, you need to stay behind the ball. I hate the term “pushing the ball” too, because it isn’t correct. But the key concept they are trying to get across is that you want your fingertips to be behind the ball, all the way through your release of the ball, this is also why we keep a stiff wrist.
I think to get more consistent with a knuckleball, it is important to practice the release with your hand to get the muscle memory down in your fingers, you will need to practice with the arm motion too, to put it all together. But it is important to spend some time just slowing down the release with the ball in your hand, watch how your fingertips guide the ball out and how your remaining fingertips are only there for stabilization. Lay in bed and toss the ball up with a knuckleball grip trying to get no spin. Even toss a few underhand knucks up in the air and catch them to get a feel for how your fingers effect the spin on the ball. Playing a game of hot potato with a knuckleball grip can be a good way to practicing the release of the ball as well, just be careful not to get your fingernails busted.
You can get creative with these, but you really need to focus on just the release of the ball for a long time to get consistent, it’s good to keep reinforcing that muscle memory all the time too, even when you feel consistent with your knuck.
To work on the velocity aspect of it, and the “fastball mechanics” (I’m guessing you did some research on Dave Clark?), playing long toss with a knuckleball is a good way to do it, in addition to playing catch with them from your knees and of course, working in the bullpen.