Good surface and joint preparation will help you complete a professional-looking and long-lasting caulking or sealant job. Whether replacing old caulk or sealing a new joint for the first time, it is important to remember to clean out the old caulk from the surfaces, cracks and crevices of the areas where you will be applying the new caulk. Traces of dirt, paint, oil, and old caulk left behind will only lead to problems later on.
Use a putty knife, painter's 5-in-1 tool or another similar tool to remove any old caulk in the joint. A heat gun can be used to soften old caulk and loosen paint to make removal easier. You can use a chemical indoor/outdoor caulk-remover to remove all types of old caulk as well. The surface needs to be completely free of old caulk, peeling paint, weathered wood fibers, grease, oil, wax, dirt, rust, frost, moisture, etc. A wire brush works well to remove contaminants, and a drill-mounted wire wheel is often the best answer for cleaning dirty, unsound concrete. To remove some contaminants (like oil or grease) it may be necessary to wipe the joint down with a solvent-laden rag (letting the solvent completely evaporate before caulking). Remember, the best caulk in the world won't work if it is applied to a dirty or imperfect surface.
If the joint or crack to be sealed is 1/4" wide or wider, it is best to install a foam backer rod in the joint before applying the caulk or sealant. This will give you better results and save you money. Backer rods are generally cheaper than a good quality caulking compound, and most of the joint can be filled with the backer rod before the actual caulk or sealant is installed. It also provides better caulk or sealant adhesion in the crack or gap.