Gmail, Outlook.com, Yahoo!, and AOL all provide IMAP/SMTP access (and perhaps even POP3, though I can't figure out why anyone would still use that when IMAP is available), and even though web clients have existed for as long as I've used e-mail, I've always preferred local applications.

As for instructions on how to set up Thunderbird to use your Hotmail/Live/Outlook account: follow the "using IMAP" instructions here.

There are some security issues if you don't want your password and other data sent over the wire between your machine and server in plain-text.

You probably want to get an SSL certificate, Let's Encrypt is a good enough (and free) service for that, but depending on which client you use, you probably need to set up a way to install the certificate to the mail server. Let's Encrypt certificates have a quite short time limit (1 or 3 months, I don't remember at the moment), so a client is really needed to automate getting a new certificate before it expires.

Since I run an ancient version of OS X Server, and can't upgrade easily without losing data (the machine's a fucking unstable patchwork), I haven't been able to set up Let's Encrypt, so I set up an SSH tunnel every time I want to check my e-mail on the go. My server is also a fucking slug (1.2GHz Core 2 Duo), and it takes an age and a half to log in, and several times that to actually check my e-mail.

A Pi should absolutely be okay to at least play around and learn with. Do be aware, though, that if it breaks down for any reason, not only won't you be able to access your e-mail, but there'll be nowhere for e-mail to be delivered to (and originating servers will stop trying to send those e-mails after a while of your server not being available).

I always prefer POP3, even with my own domain and my own mail server. :-P

Care to tell me why?

As for my reasons:
I like being able to organise my mail in a folder structure, and as far as I've seen, POP can't organise on the server, only locally. I also haven't seen POP copy messages from the client to the server (useful when switching servers/providers) - but I may be mistaken on that. So, with IMAP, I have copies of the mail storage on the clients as well as on the server, so there are backups if any point fails.

I do not want to keep private data at remote servers.
If a mail is so important that I want to access it at multiple places, I just re-send it to myself, or make a copy in my portable storage.

If you can't trust the server to hold it, how can you trust it to temporarily store it?

It is what POP3 mail server supposed to do: temporarily store it.
Once I got the mail, I want it to be deleted permanently at remote server.

Sure, but that's not what kohlrak implied. If you think the server company is "shady" enough (for lack of a better word I can't recall now) that you don't want them storing a backup for you, why wouldn't you think that they've made a copy for whatever ill intent they might have when the messages first arrived?

All-right. Fair enough.

I also have to ask who's hand cannot be trusted. For example, I've actually had microsoft try to spy on me and others with a man in the middle attack, provided by the United States government.

EDIT: To be fair, they only saw WHO was emailing me, instead of directly setting up a man in the middle for my emails themselves. The process they went through would be similar, and presumably the US government can do it at the drop of a gavel.

Thanks for the recommendations, everyone! There are a few too many for me to quote and reply to individually without creating bloat, but I've read them.

Thanks to the quote limit and the auto-editor, you would't be able to anyway. XD

I'm another vote for Gmail/Inbox.

Exactly, this is what I've been doing for years now and it works like a charm.

A quick check shows kohlrak.sytes.net sitting on a Comcast cable modem. If you;re trying to send mail from there, you're going to get blocked as end user cable modems shouldn't be sending emails. In theory you should be sending it to your ISP's email server and then having it go out from there.

In theory at least.

Your IP address is blocked on a few lists for such a setup. In ours, it's a static +1 on a 5 point scale for determining spam.

The domain used wouldn't affect anything. It really looks like the IP address and the fact that it;s a cable modem to be the issue.

Your MX record is showing you have it set up to send email via the no-ip.com mail servers. You may have better luck with that if you;re able to. I've never used no-ip.com. Maybe someone else can chip in.

Hope this helps