First step is to get a domain name. Get a name that reflects both you and your niche. If the blog is for a business then you might already have a domain name, if not find one that most closely matches the name of your business.
Next up is hosting. You will need a host for your blog. Fortunately for those just starting out, hosting a small blog is cheap. The domain name provider might also host websites. Avoid getting dedicated services and high end hosting packages. Look for something that is a starter or baby package.
Now you can setup your blog. The easiest way is to choose a CMS like WordPress. Most hosts support one click installations of whatever CMS you choose. Choose a CMS and then using the quick install of the host, install it and you are ready to go.
You can customize the look and feel of the CMS with a theme. There are plenty of free themes available for every type of blog. If the blog is for your business find one that is more business focused and that can be customized more to match the color scheme of your business.
Your blog is ready. Now all you need to do is populate it with posts.