The impact of calcium sulfate (anhydrite and desulphurization gypsum) on the hydration of calcium sulfoaluminate (CSA) cement, such as compressive strength, drying shrinkage ratio, hydration heat of early stage and microstructure evolution, was measured. Results reveal that, the early hydration of CSA cement is accelerated by the addition of calcium sulfate, resulting in higher 1day compressive strength and lower drying shrinkage once the dosage is less than 20% (mass ratio). However, with the increasing amount of calcium sulfate, 3day and 28day strength is prominently decreased, and even the retraction of strength happens when doping with excessive calcium sulfate. Compressive strength has no direct correlations with ettringite amount, while it has positive correlation with the alumina gel amount. Additionally, to achieve CSA cement with better properties, it is very promising to replace anhydrite with desulphurization gypsum.