I know when I first started out I didn't have much idea of what I was doing or where I needed to put my books, so this page is just a little list to help others in the same situation.
The Big Three
Amazon KDP - you use your Amazon ID to sign in and you can start uploading books straight away. They have a well ordered help section which will show you how to go about it.
Amazon also offers KDP Select, which gives various extra promotion options and the possibility of a share in the monthly global KDP fun, however, it means you cannot publish anywhere else. I object to this on principle.
Smashwords - this is an eBook distributor which will send you books to Apple iBooks, Barnes & Noble, Kobo, Flipkart, Oyster, Scribd, the Diesel eBook Store, Baker & Taylor's Blio and Axis360 (libraries) and more. They have a starter page here to give you all the help you need.
Google Play - these guys are finally getting there, their interface used to be terrible and it's still more complex than Amazon and Smashwords, but it's getting better. They have a checklist here on how to set up your account and sell books.
Of course there are many other small sites as well, but these are the big names.
If you have any questions or suggestions for additions to the list, please feel free to drop me a comment.