If your goal is to gain mass, as long as you don't do too much you should be ok... Once the duration and/or intensity of your cardio work (swimming is cardio and resistance) you're doing becomes too long/intense, it can interfere with your strength and size gains. I'd say to opt for a couple swims per week at under an hour in length, and maybe use it primarily for active recovery if you're trying to bulk :)