There are quite a few ways of going about it. The absolute cheapest option is to run a free blog on something like Blogger.com or Wordpress.com. They're free but you don't quite have the control over everything that you would if you ran it yourself. Also, free options very rarely let you use your own domain name, so the URL is usually "your-website.wordpress.com" or something like that.
If you want your own URL, you need a domain name. To get a domain name, you need to buy it from a registrar. I use and recommend namecheap.com. I'm also told that hover.com is a good one. You buy your domain name from a registrar and typically renew it on a yearly basis.
Now that you have a domain name, you need to point it somewhere. Some blogging services will let you point a domain name at them. Tumblr lets you do this for free, Wordpress.com charges something like $13/domain/year. That way, you're using someone else's blogging service but you have your own URL. You don't have to worry about some of the more fiddly bits of hosting, and you still have your own URL, but you still give up some control to the blogging service.
If you don't want to go through a blogging service or you want more control than they afford, you need to find a web host. This is where you pay someone else for them to let you use their server. You can run your own software on it, like Wordpress (.org, this is the self-hosted version of Wordpress.com) or Movable Type, or Drupal, or any other mess of options. You can run a hand-edited collection of HTML files if you want! You also get other features like being able to have your own email addresses and FTP server. The host usually imposes limits on hard drive space, bandwidth used, databases in use, and number of domains or subdomains. Sometimes, they're particular about content. Some web hosts don't want you to host your own videos. Using your own web host is pretty DIY. They've often got tools to setup blogs but if you run into a problem with the blogging software, you're kind of on your own.
There are a ton of options for web hosts. 1and1.com is an obvious one. I use Lithiumhosting.com. Other people are happy with asmallorange.com. The trick is to find one whose limits you're comfortable with and doesn't cost a ton. Since there are so many options, shop around. For just a personal blog, you probably want something on a shared hosting account with some fairly low limits. And most web hosts will help you get setup, particularly with pointing your domain name to their service. That part can be less than fun if you're new at it.
And if you want to go completely bonkers, you host your website yourself. This is the completely DIY option and usually not worth it for a personal blog. But if you're into servers and stuff, it's probably the most fun option!