The fastest part of your golf swing should always be at impact.

If you are accelerating into the ball, you will deliver maximum club head speed, and therefore achieve maximum distance on your golf shots. Accelerating too early on the downswing will result in the club head decelerating as you connect with the ball, and therefore not delivering the maximum club head speed into the ball resulting in a loss of distance on your shots.

A great drill to use to experience exactly where your club head is at it's fastest is what I call the 'whoosh' drill. Turn your club upside down so that you are holding the heavy head end. Hold the shaft as though it were the handle of your club and swing back making your usual golf swing. Swing down and listen for the whoosh. If you did not hear one keep repeating your swing, working on swinging faster. Once you are swinging fast enough you will hear the whoosh noise.

Now notice where you heard the noise. If it was to the right of the bottom of your swing, you are accelerating too fast too early on your downswing and releasing all of your speed before you are at the ball. Work on making the whoosh noise later, when the club is at the bottom of your golf swing so that you are delivering maximum club head speed into the ball. When you feel you are making the whoosh at the bottom of your downswing then turn the club back to it's normal position and begin to replicate the whoosh swing. Listen for the noise at the bottom of your swing and you will be delivering maximum club head speed into your golf shots.

With practice you will become far better with this and you will see an increase in the length that you hit the golf ball.