Based on the Fick’s second law, a diffusion reaction equation of sulfate in concrete piles was established. With the method of separation of variables and Danckwerts, according to initial conditions and boundary conditions, the diffusion reaction equation was deduced. Furthermore, considering the effect of pore filling and crack on diffusion coefficient, an effective diffusion coefficient model was proposed to show the variation of the diffusion coefficient during the degradation process of concrete piles. Sulfate concentration profiles were obtained by the solution and the effective diffusion coefficient model, which agrees well with the experiment data to verify the validity of the proposed analytical solution. Case study shows that the inhibition effect of pore filling on sulfate diffusion is remarkable. The development of the crack promotes the diffusion reaction of sulfate. With watercement ratio decreasing, the invasion depth of sulfate decreases, as well as the concentration of sulfate in concrete piles.