What is the difference between a tennis racquet's replacement grip and an overgrip?

A replacement grip takes the place of the original grip that was attached initially, while an overgrip is placed over the existing grip on the racquet.

Should I use an overgrip?

It creates a thicker grip for comfort and absorbs sweat. Overgrips are an alternative to buying a replacement grip for your racquet.

Is there a difference between tennis, badminton and squash racquet grips?

There is not a lot of difference, and usually grips can be used on all racquets. However, it is always best to check the packaging for recommendations. Badminton rackets are typically thinner and require smaller grips than tennis rackets, which is why it's important to find a size that fits the handle diameter snugly.