SSH connections on Mac OS X are sometimes very slow (long delays) although it's instantaneous on Linux. Modifying an option in the configuration files fixes this injustice.
sudo vi /etc/ssh_config
Replace this line:
# GSSAPIKeyExchange yes
by:
GSSAPIKeyExchange no
Don't forget to remove the sharp!
sudo vi /etc/sshd_config
Replace this line:
#UseDNS yes
by:
UseDNS no
Again, don't forget to remove the sharp. That's it !
May be you're not enthusiastic about hacking your SSH configuration files. Another solution is to add the IP addresses you're going to connect to (or which are going to connect to your mac) in /etc/hosts.