Yes, an SSL certificate (both shared and private) will slow down the loading speed of your website. This is because the connection has to be encrypted/decrypted on the server and on the client end, which takes time.
The computer you are using will also affect loading speed as will flash animation, banners and anything else you have put on your website that takes up computer memory.
However, the trade-off is that having an SSL certificate on your website gives your customer as sense of safety and security.