Thanks to David Green for flagging up this whoishostingthis.com infographic on Twitter - I found it useful because I tend to use a mixture of JPEG and PNG with saving images, with not much understanding of why I'm choosing what I'm choosing...
JPEGs reduce the size of an image by compressing it - making the image less detailed and so the file-size smaller, but effectively doing so in a way which the human notices least. This is important for web-use as everything each viewer of your website has to download - which is to say, have appear on their screen, rather than save to their computer - takes up bandwidth, and bandwidth costs money. One of the reasons I switched over to Squarespace from Wordpress.org is the unlimited bandwidth meaning I don't have to worry about upgrading my package to accommodate more.
PNG files can be transparent - they don't have to have a white (or black, or any other colour) background. This is truly useful because they can 'sit' on top of any background - so for organisatinal logos for example, it's essential to be able to drop them in over any kind of poster or slides or publicity materials, without an ugly white background delineating the logo from the rest of the content. PNG files generally take up more space than JPEGs but keep their quality better by not compressing the file in a lossy way. This is useful for something like uploading an image to Twitter; Twitter compresses the image, so an already compressed image can start to look really quite rough by the time it a Twitter user sees it.
GIFs I use a lot but never really have cause to make. They generally take up the least space and work better for created graphics rather than photographs.
After the recent posts on which format to choose when saving presentations, and which sizes to use when saving social media images, it seems only right to complete the set with one on how to save pictures too! So courtesy of whoishostingthis.com, here we are: