Software RAID – Pt. 1

I attended the monthly Central PA Linux Users Group meeting again this month.  Doug talked about software RAID in Linux.  He went through all the trouble of building a 3TB array during his talk and this sent me scouring for an virtual disk solution.

I remembered a ZFS tutorial which made use of several file backed disks to allow experimentation with the file system, to learn the commands, etc.  (ZFS Tutorial 1)  If you click through to that tutorial, you’ll see they make use of Solaris command ‘mkfile’.  As this is not included with Linux, I had to dig around for another way to make a disk like file.  Read enough books on virtual machines and you’re bound to remember something about using ‘dd’ to create virtual disks.  That problem solved.

The next issue I ran into was that ‘mdadm’ did not like these raw disks.  So I tried a couple things until, after too long, I found prior art using loopback devices.  That now fixed, viola, RAID-6 array built from 6 100MB virtual drives.  Heck of a  lot easier to blow away and rebuild.

After figuring out the whole process now, I finally know enough search terms to get Google to give me just the result I wanted from the start.  This blog post (Stupid RAID Tricks with EVMS and mdadm) gives a step-by-step walk through of setting up and managing RAID.  I haven’t had time yet tonight to go through it, but I will do so and post back here with my own take on it.  That post is 4 years old (some recent comments though) so I’m certain some things have changed since it was originally posted.


I said a couple days ago that I had killed off OpenFiler and went with FreeNAS.  A couple of things changed since the post about getting OpenFiler working as a SAN.

I would like to be able to add a second VM Host and having external storage is pretty much essential for that to be practical.  So with this in mind, I went to Newegg and picked up an Micro-ATX Atom motherboard, small but nice case, some RAM, and a good Intel Gigabit NIC.  (Already had the drive I was going to use.)  I put that all together and went immediately to FreeNAS.

I installed FreeNAS 0.7.1 onto a USB stick (from the CD, didn’t use a second computer) and hooked everything up and tested it out.  It apparently worked since I stuck with it.  The USB boot was important because I didn’t have a 2nd HD lying around and it seemed like a huge waste.  Even if I did partition, I have no plans of even filling the dedicated drive, let alone the remainder of the OS drive.  OpenFiler doesn’t have this capability, so barring lack of functionality, FreeNAS was the way to go.

Now for my FreeNAS setup, I simply added the 250GB drive and since ZFS was an option, I just a single stripe ZFS file system.  (I wasn’t going to get any redundancy anyway so I figured no harm and potential upgrade path.)  I configured iSCSI (plenty of guides out there for that) and bada-bing, iSCSI hosted virtual machines.

Lots of Changes

Wow, didn’t realize it’s been a month now since my last post (until some prodding).  And looking at my last post, there’s a lot that’s changed since then.  I’ll try to make a couple posts over the next several days to try to bring the blog up to speed on the current system status.

Here’s a quick list of the things off the top of my head:

  • OpenFiler is dead and gone.  FreeNAS all the way.
  • Installed Citrix XenServer to VM Host (put ESXi on USB)
  • Purchase of a Netbook (no luck with Hackintosh’ing it)
  • Progress studying for RHCE

git server

Spent the day figuring out git and setting up a server on my ESXi server.  I created a new VM and installed Ubuntu 9.10 Server 64 on it.  I own the book Pragmatic Version Control with Git from Pragmatic Programmers on PDF so I worked through that while the server was installing and getting packages installed and whatnot.  For the actual configuration of the server I followed the guide from Andrew Dunn Install (Set Up) Git and Gitosis on Ubuntu.  It was pretty straightforward.  I opted for gitosis in the event I need to add anyone to the setup and since it wasn’t very difficult to begin with.

I was a little concerned with the server names being what they are and being able to access the repository remotely but I realize, with git I don’t really have to worry about that.  (Until I get another contributor and we’re actively pushing up changes, I suppose.)

OpenFiler SAN

After about a week of messing around with it, I finally got an OpenFiler system setup as a guest OS inside ESXi and sharing via iSCSI.  Honestly, I don’t even remember what finally fixed the issue, but I was finally able to see the iSCSI share from vCenter and configure it as a datastore.

I added a 250GB (base 10) SATA drive to the ESXi system originally thinking I could use it as a RAW disk, but soon found that not to be the case and confirmed that.  So I created an additional datastore, then added a 230GB (base 2) virtual disk to the OpenFiler system.  After that, I just stepped through some instructions on how to setup iSCSI on OpenFiler and using it with ESX.

There were a couple issues I do remember having.  One was some confusion about the IP addressing scheme used in the tutorial.  The tutorials tended to keep the iSCSI completely internal, which isn’t what I really wanted (I may add another host), and I think this is where the confusion came from.  After I confirmed that iSCSI was working within the confines of the machine, I was even able to connect from my laptop.

Also, I forgot every time I added an entry to the Network Access Configuration, to set it to Allow in the Network ACL for that iSCSI share.

I should probably repeat the process independently to solidify it in my mind.  Perhaps a project from tomorrow before I start adding VMs the “SAN”.  Maybe even stick in the 160GB PATA drive I have lying around and see how it plays with OS X.

I went ahead and made a diagram of the network I’d like to put together.

The firewall box has been ordered.  I’ll put that together and get it setup once it arrives.  I’d like to install the FreeSWITCH package as well and see whether I can get a decent soft phone working.  By that time I should have the parts to have an acceptable OpenFiler system setup and a single ESXi machine.  The dream is to be able to farm out jobs to a couple systems as needed and spin up instances of custom images locally.  Say for DVD encoding, DVD ripping, data mining, etc.

Computers are what I love.  This can be somewhat of a problem since that’s a rather broad topic.  I’m hoping to use this blog as a record of the various projects I undertake for posterity and as a measure of progress.

For the moment, I’m just going to list a bunch of projects I’d like to work on.

  • Setup VMware system
  • OpenFiler NAS/SAN (attached to VMware)
  • Setup pfSense firewall
  • Personal Cloud/Cluster
  • Cable the house – so I can take the noisiest boxes out of my room and theoretically for VOIP
  • Setup FreeSwitch phone system (I don’t have the need for a “land line”, I just want to figure it out or possibly build myself a private Google Voice so they’re not controlling EVERY part of my existence)
Things I’m dabbling in at the moment include:
  • Reading Scott Lowe’s Mastering VMware vSphere book
  • Reading pfSense: The Definitive Guide by Chris Buechler
  • Working my way through an A+ certification book
  • Working through a Data Mining book
  • …non-computing books
Hopefully some of this will come to fruition.

Hardware Issues

A little over a year ago I finally bought the parts to build myself a NAS.  My plan was to install FreeNAS and use ZFS.  I had hardware troubles from the get-go and seemingly inconsistent results.  I was never able to get FreeNAS to a point that I liked (I think I had issues doing the install and/or CF card) and eventually had OpenSolaris running on it, thinking I’d get at least half of what I wanted.  But that wasn’t good enough and eventually the project was sidelined and my current drives sat, and my new drives sat until I found a deal on a SATA enclosure and just loaded them in there and hooked up straight to my laptop.

Today’s trouble is with the original hardware for the NAS which I’m repurposing as a virtual machine testbed.  I seem to be plagued with loose connections or something.  A few days ago when I went to install ESXi the system refused to boot from the CD.  Went to the BIOS and there was no CD drive, whereas a week before, as was well, and I hadn’t touched the hardware.  I remember this also happening constantly during my initial NAS setup.

So what got this post started was ESXi not liking the built-in network card, I went out and purchased an add-on card which was coming up in searches and appeared to be compatible, and now it is not.  So I’ll probably be returning this one (which by the was already opened by someone) and head out to Best Buy since I feel like they’d have the better quality cards in stock.