Archive for the 'Linux' Category

Throw some Rocks at it!

ganglia
One of the parts of my day job is dealing with and managing our HPC cluster. This is an 8 node Rocks cluster that was installed maybe a week after I started. Now I was a bit green still at that point and failed to get a better grasp on some things at the time, like how to maintain and upgrade the thing, and I have recently been paying for that :-)

Apparently, the install we have doesn’t have a clear-cut way to do errata and bug fixes. It was an early version of the cluster software. Well, after some heated discussions with our Dell rep about this, I decided what I really needed to do was a bit of research to see what the deal really was and if I could get us upgraded to something a bit better and more current.

Along came my June 2009 issue of The Linux Journal which just happened to have a GREAT article in it about installing your very own Rocks Cluster (YAY!). Well, I hung on to that issue with the full intention of setting up a development/testing cluster when I had the chance. And that chance came just the other day.

Some of you probably don’t have a copy of the article, and I needed to do some things a bit different anyhow, so I am going to try and summarize here what I did to get my new dev cluster going.

Now what I needed is probably a little different that what most people will, so you will have to adjust things accordingly and I’ll try and mention the differences as I go along where I can. First off, I needed to run the cluster on RedHat proper and not CentOS, which is much easier to get going. I also am running my entire dev cluster virtually on an ESX box and most of you would be doing this with physical hardware.

To start things off I headed over to The Rocks CLuster website where I went to the download section and then to the page for Rocks 5.2 (Chimichanga) for Linux. At this point, those of you who do not need specifically RedHat should pick the appropriate version of the Jumbo DVD (either 32 or 64 bit). What I did was to grab the iso’s for the Kernel and Core Rolls. Those 2 cd images plus my dvd image for RHEL 5.4 are the equivalent to your one Jumbo DVD iso on the website that uses CentOS as the default Linux install.

Now at this point, you can follow the installation docs there (which are maybe *slightly* outdated(?), or just follow here as the install is pretty simple really. You will need a head node and one or more cluster nodes for your cluster. Your head node should have 2 interfaces and each cluster node 1 network interface. The idea here is that your head node will be the only node of your cluster that is directly accessible on your local area network and that head node will communicate on a separate private network with the cluster nodes. With 2 interfaces, plug your eth0 interface on all nodes, head and cluster into a separate switch and plug eth1 of your head node into your LAN. Turn on your head node and boot it up from the Jumbo DVD, or in the case of the RHEL people, from the Kernel cd.

The Rocks installer is really quite simple. Enter “build” at the welcome screen. Soon you will be at the configuration screen. There you will choose the “CD/DVD Based Rolls” selection where you can pick from your rolls and such. I chose everything except the Sun specific stuff (descriptions on which Rolls do what are in the download section). Since I was using RHEL instead of CentOS on the jumbo dvd, I had to push that “CD/DVD” button once per cd/dvd and select what I needed from each one.

Once the selections were made it asks you for information about the cluster. Only the FQDN and Cluster name are really necessary. After that you are given the chance to configure your public (lan) and private network settings, your root password, time zone and disk partitioning. My best advice here would be to go with default where possible although I did change my private network address settings and they worked perfectly. Letting the partitioner handle your disk partitioning is probably best too.

A quick note about disk space: If you are going to have a lot of disk space anywhere, it’s best on the head node as that space will be put in a partition that will be shared between compute nodes. Also, each node should have at least 30gb of hdd space to get the install done correctly. I tried with 16gb on one compute node and the install failed!

After all that (which really is not much at all), you just sit back and wait for your install to complete. After completion the install docs tell you to wait a few minutes for all the post install configs (behind the scenes I guess) to finish up before logging in.

Once you are at that point and logged into your head node, it is absolutely trivial to get a compute node running. First, from the command line on your head node, run “insert-ethers” and select “Compute”. Then, power on your compute node (do one at a time) and make sure it’s set to network boot (PXE). You will see the mac address and compute node name pop up on your insert-ethers screen and shortly thereafter your node will install itself from the head node, reboot and you’ll be rockin’ and rollin’!

Once your nodes are going, you can get to that shared drive space on /state/partition1. You can run commands on the hosts by doing “rocks run host uptime”, which would give you an uptime on all the hosts in the cluster. “rocks help” will help you out with more commands. You can ssh into any one of the nodes by simply doing “ssh compute-0-1″ or whichever node you want.

Now the only problem I have encountered so far is I had an issue with a compute node that didn’t want to install correctly (probably because I was impatient). I tried reinstalling it and it and somehow got a new nodename from insert-ethers. In order to delete my bad info in the node database that insert-ethers maintains I needed to do a “rocks remove host compute-0-1″ and then “rocks sync config” before I was able to make a new compute-0-1 node.

So now you and I have a functional cluster. What do you do with it? Well, you can do anything on there that requires the horsepower of multiple computers. Some things come to mind like graphics rendering and there are programs and instructions on the web on how to do those. I ran folding at home on mine. With a simple shell script I was able to setup and start folding at home on all my nodes. You could probably do most anything the same way. If any of you find something fantastic you like to run on your cluster, be sure to pass it along and let us know!

CentOS 5.4 The Real Deal

tpt23
I promised that I would try the full install version of CentOS 5.4 desktop on my thinkpad and didn’t want to disappoint, so here it is…

I actually had a really hard time with this one. That is not to say that I believe that there is an issue somehow with CentOS, but certainly something odd with the lappy at the very least. For some reason, no matter how hard I tried, I could not get the installer to run correctly on the dvd. Or more correctly put, on any of several dvds. The installer would just randomly crap out in different places. Finally I just tried an old CentOS 5.3 dvd in an external dvd player, and that finally did the trick. Besides, it only takes one quick “yum -y update” from there and you’re at 5.4 anyhow.

Like the live version (or should that be the other way around) the full install of CentOS 5.4 is quite good looking and very snappy. It uses the gnome desktop and has all the goodies you would expect from a full blown enterprise desktop. It also carries, smartly, the software that I personally use in a day to day basis - firefox, thunderbird, openoffice, etc.

You know, I almost dislike reviewing this particular distribution because there is nothing particularly exciting about it other than it does what I like for a business desktop distribution and does it quite well. The same goes for the server install, both available from the same dvd, you get a true, reliable enterprise class Linux that “just works” ™ like it’s supposed to. I guess, that is the exciting part. You get a good Linux without having to tweak and mess with a bunch of things.

This sure isn’t a *home* desktop Linux. There’s no easy support for multimedia, so I wouldn’t go springing this on mom and dad, but for a business desktop, you just can’t go wrong here. And just FYI, there are plenty of good instructions on how to get your media on within a quick google search.

As unexciting as this sounds, I am still going to get a more permanent desktop install of this somewhere in my house. Just like I run some servers at home on CentOS, it sure couldn’t hurt to have a really stable and quick workstation somewhere within easy reach too!

Year of the Linux Desktop? Not!

20090307-mint
Let me preface this by saying that I use Linux almost exclusively as my desktop at home and at work (I have a mac too). My wife uses Linux exclusively as her desktop. Many of my friends and coworkers use Linux as their default desktop as well.

That being said, is 2009 the year of the Linux Desktop? No way, and 2010 isn’t going to be it either.

Now many people are going to say that Linux still has a way to go before it gets it’s “year of the desktop”. Opinions differ, but people point out things like hardware compatibility issues, lack of a unified desktop, ease of OS install, ease of program/utility install, windows program compatibility issues, and on and on.

What I would like to say is that none of that amounts to a hill of beans. The year of Linux on the desktop happened long ago when I was able to start using Linux full time instead of anything else. And not just I, but countless other users as well. You see, there might have been some issues along the way, things that got fixed or better as time went along, but all in all, I have been able to use Linux on my desktop and function not only in a home, but corporate environment just fine for almost 10 years.

Don’t get me wrong, things have certainly gotten easier lately. We now have distributions like Ubuntu which is practically brainless to install and looks and runs fantastic. This was not the case way back when, but I was still able to get the job done. Then again, I can remember a lot of things in other more popular operating systems that ran rather badly back then too :-)

We have great utilities now with installers that are light years ahead of other OSes. We have almost fully Microsoft compatible office suites (or in some peoples opinions better). We have the whole gamut of media players, productivity utilities, and yes, even games, many of them really fantastic! Having so many “cloud” utilities is helping even more.

I think Steven J. Vaughan-Nichols hit the nail on the head when he wrote it’s not the year of the desktop because we’ve already had it. Most people just didn’t notice.

Ubuntu 9.10

tpt23
Yes, another in the junker series on my T23. I *finally* got around to playing a bit with the new Ubuntu 9.10 Karmic Koala. I know, based on the date of the posts, that it’ll look like I am doing a distribution a day (magic of dated publishing), but I have had this running a few days now and I really dig it.

The installer is very fancy looking now and I really like the new loading logo graphics. They have this cool little cylon slider that goes across the screen - much slicker than the big blocky one on previous versions. All my equipment was detected correctly and, I should add, this is the norm for any recent version of Ubuntu, and they should be praised for that. I have come to expect nothing less from them.

The default background is *still* brown. Ick. They *do* have a much better selection of other wallpapers to choose from though. Honestly, if the most you have to complain about is the default brown background, things are pretty good. And things here really are pretty good!

ubuntu
Ubuntu still comes with all the software it always has - the things you need to get the job done. And what is not there by default is only a click or two away. Ubuntu has come to be the go-to distribution for a nice workable desktop Linux. It’s clean, up to date, and perky too. What more would you expect? Well, maybe a different color wallpaper (I like the stones one). :-)

CentOS 5.4 Live

tpt23
In my continuing saga of Linux distributions and testing on my trusty, crusty Thinkpad T23, I tried out CentOS 5.4 Live. Now many of you already know that I have developed a sort of love affair with CentOS as a wonderful alternative to paying for enterprise class Linux. Then again, I am RedHat certified, so I like to be able to keep sharp by using similar products on my home and work servers.

Now I will have to say that my original intent was to just install CentOS 5.4 proper on this laptop, however, at the time I was grabbing the iso, I was away from the lappy and forgot whether or not it had a dvd drive in it (it does). So, while looking at the 6 cd’s I was going to have to grab and burn, I noticed the “Live” cd. I thought to myself that I should just grab that and install from there. After all, installing from a live cd has become quite commonplace these days hasn’t it?

Even on a live cd, i noticed that CentOS booted up pretty fast. I was excited to try it out as I recall my RHEL and CentOS 5.3 desktops were quite snappy. Well, it wasn’t long before it was running and I was horrified at what I saw. CentOS refused to properly autodetect my display, instead providing me with some frightfully blocky looking 800×600 default display. Ick. I was pretty sure right off the bat that this was going to be a “quick” trial :-)

CentOS

CentOS


I decided to plod through this and actually give the thing a real try anyway, so I set out to fix the screen resolution. The fix was quite simple. I did a quick system-config-display and picked a 1024×768 lcd screen, millions of colors and then whacked control-alt-backspace to restart X and violla, things were looking considerably better. In fact, the default desktop, under a decent color and resolution, is quite pleasant to look at and work on although I did find the default fonts not *quite* as smooth looking as those on Ubuntu or Mint. Not that they were bad looking, just perhaps not as polished.

All that aside, once things were running, they ran well, very very well, and fast too. CentOS 5.4 is hands down the fastest running live cd distribution I have ever used. The desktop is really snappy. The cd access is very quick compared to other live cds I have used. One of the most impressive things is that programs like firefox (the default webbrowser) started off that cd *faster* than it did on a full install of mint. At least it felt faster.

The other really fantastic thing about CentOS 5.4 Live is the selection of programs available on the cd. It has Firefox for a browser, Thunderbird for email, OpenOffice and just about everything else *I* use on a day to day business basis. Of course there are no media codecs, outside of the free ogg variety, but hey, this is an enterprise desktop right?

This brings me to the installer. My intention was to try a full CentOS 5.4 install here and especially when I wanted to play a video or two. I figured it would be a lot easier to get some necessary codecs installed on a full version than a live cd. Well, the problem is there is no installer on the live cd. They expect you to install from the dvd or the cd set and the live cd is just that, a live cd. That being the case, I have put it in my to-do list to do just that, and install a full version later on for some testing, but you can bet that I am gonna keep that CentOS live cd close at hand as I can see that being a fantastic resource for a lot of things like filesystem forensics, fixing broken servers and even anonymous/secure access from other machines (bring your own linux with you).

Check it out. If you are at all familiar with RedHat/Fedora style of things, or you are looking for a nice fast live cd, give this a look over - you’ll like it.

Ressurection of an old friend

tpt23
I think one of the things that is going to help me post a bit more this month is my ThinkPad T23. This was my most favorite laptop some 4 years ago maybe and the display just stopped working one day. I held onto it for a few months and then landed my new job and moved where I pressed it into service as a server to run my home computer infrastructure. It performed admirably under Ubuntu 6.04 and 7.04 for some 2 and a half years hardly being touched until I recently replaced it with some real server hardware.

Sunday night, I finally dug the thing out from the shelf underneath my TV for the first time since I initially set it up. I blew the dirt, hair and dist off of it and, just for giggles, opened up the lid and hit the power button. To my complete surprise, the display fired right up and my old Ubuntu 7.04 server started coming to life. Don’t ask me why the display started to work after 3 or 4 years, but hey, when in Rome right?

I decided to take advantage of the moment and I grabbed for my Mint 7 cd and ran a quick install. I figured I was going to have to format the computer sometime anyway before I threw it out or gave it away and this would allow me to play a little. Well, in no time I have Mint running on this little beastie and it looks and runs very nice! The only real problem I have is that the pcmcia network card I have does not seem to support wpa. This means I will have to find a long ethernet cable somewhere in order to play on it. Bummer.

The great part about this, though, is as long as the display keeps working, I can use this as an excuse to play with a bunch of different Linux distributions and post the results here!

National Blog Posting Month

Well, here it is, National Blog Posting Month again. I have decided to accept the challenge. I do, however,think that I am setting myself up for failure. Just curious as to how long that is going to take. :-)

Do stay tuned, though, as I will attempt to interject a few interesting things, if possible, from my many times mundane sysadmin life!

I would like to take this opportunity to challenge the other Linuxish bloggers to do the same and perhaps we can flood the market (so to speak) with some interesting Linux/FOSS/BSD content this month! You can do it!

Who needs work?

I remember several people at OLF recently telling me that they were hurting for / looking for some work. Well, I get hounded by headhunters quite often and would gladly pass stuff on to those people who are interested, but I need to know who you are :-) Send me an email at linc dot fessenden at G mail dot com and let me know.

Retail w/ Linux

It’s been a while since I last posted. I have been pretty sick for a week or so.

A couple weeks ago I had to make a quick run to Microcenter and, while there, I spied these on their discount software shelf for $3 a piece. Nice to see Linux disks in places other than stuck to a magazine - even if it IS the discount software rack :-)

ld1

ld2
ld3

On Web Video

When I decided I wanted to do this new show thing and do it in video, I didn’t know what I was really in for. Then as I started playing a little with video and wanted to stick a snippet on the web, I was sure I was in trouble.

This is a collection of quick notes on how I did it.

First, I grabbed some video off of my camera. Now the camera records to avi but does it in high-def, which is a little large for what I wanted, so I ran it through tablet-encode, which is an mencoder wraparound script that generates small-enough video that works on the n770/800/810 devices. This worked like a charm and a line like so: (code is all one line)

tablet-encode –preset=best -2 file.avi newshow.avi

got me an avi file that was 400×200 30fps.

Next I wanted to get that on the web and viewable. A quick look through some internet available howtos revealed that an flv (flash video file) was the way to go there. So, off I went to create flv from avi. After some serious digging I was direted, once again, back to mplayer/mencoder. (code is all one line)

mencoder -forceidx -of lavf -oac mp3lame -lameopts abr:br=56 -srate 22050 -ovc lavc -lavcopts vcodec=flv:vbitrate=250:mbd=2:mv0:trell:v4mv:cbp:last_pred=3 -vf scale=360:240 -o $1.flv $1

Then was the tricky part. I wanted to just use an embed or object statement in some html to stick that video in the page, but that would not work at all. It took me hours of looking through people’s bad documentation on the net to discern that what I really needed was a flash player in order to play flash video. Another quick search revealed the free-enough JW FLV Player.

Once you have that and get the “player-viral.swf” or flv player on your webserver and accessible, there is only one more little barrier. It’s nice to have a picture to have at the front of your video instead of a black screen (before your play the video). Once again, mplayer: (and once again, all one line)

mplayer -ss 10 -nosound -vo jpeg -frames 1 newshow.flv

Which will create an image named 000001.jpg or some such. I just renamed it to newshow.jpg.

Now with all that, I can put my flv and jpg somewhere I can get to, and do an embed statement in a webpage like so:

< embed
src='player-viral.swf'
width='400'
height='240'
bgcolor='undefined'
allowscriptaccess='always'
allowfullscreen='true'
repeat='none'
flashvars='file=newshow.flv&image=newshow.jpg'
/>

Obviously the lessthan sign and the embed command should go together but, alas, it will not let me post the code any other way :-)

« Previous PageNext Page »