It is only senseless if you do not believe that moral codes, and civility and ethics, are not items of value for a society...
Morality gives rise to the very principles of freedom of speech...

You have already limited the freedom of speech with the introduction of moral codes.

Rights of free speech, framed in the context of 'human granted rights', is oppression and tyranny, waiting to happen.
That's the beauty of always having a 'higher power' (whether you are religious or not).
Morality can be viewed as one such higher power, (if you are not religious) even though it's been derived from man.
It becomes bigger than man. It makes everyone accountable.

Again, it requires an agreement to imposed limitation and defer responsibility to a higher power.

This was a really good post :)

Edit: I only read my section :P