Welcome to the BigFix Blog

BigFix Power Management is part of this year’s West Coast Green, the largest green innovation, building, and design tradeshow that highlights some of the greenest inventions and ideas in the industry.  It is happening October 1-3 at San Francisco’s Fort Mason Center.

125 Speakers

80 Education & Networking Sessions

“While it’s not always the sexiest side of security, patch management is one of the most critical aspects of an organization’s overall security program. Unfortunately, despite vendors working diligently to combat vulnerabilities with patches and enterprises incorporating them, those efforts are often in vain because the end user doesn’t apply them. These days, leaving patch installation in the end user’s hand is as safe and effective as installing a top-of-the-line security system at a bank but leaving open the vault door.”

– Amrit Williams, BigFix CTO and host of Beyond the Perimeter

With just a month left for nominations, BigFix is adding a new prize to the 2009 System Administrator of the Year contest – a progressive jackpot up to $5,000. The prize will start at $2,000, with $200 added to the pot for every 10 new nominations. The final prize will be split between the winning sysadmin and his or her nominator. In the case of self nomination, the entire cash prize will be awarded to the sysadmin.

This cash prize will be the new Grand Prize, with the runner up receiving a laptop, a conference pass to the 2010 Large Installation System Administrator (LISA) Conference, Guitar Hero 5 and their choice of Wii, xBox, or PS3. The next Top 3 entrants will win Guitar Hero 5 and their choice of Wii, xBox, or PS3.

BigFix proudly sponsors the 2009 System Administrator of the Year contest, which acknowledges the under-recognized rock stars of information technology who are vital to the smooth functioning of business, government and public services.

Sysadmins are the unsung heroes of information technology, keeping computers up and running, defending infrastructures from malware and hacking attacks, and helping end-users maximize their productivity. The contest is also an opportunity for the sysadmin community to share information on best practices and new ways to deliver value to their constituencies.

A panel of judges including AEleen Frisch (author, Essential System Administration), Doug Hughes (serial presenter and guru at many USENIX LISA conferences), Ben Kus (founder, BigFix User Group), Thomas A. Limoncelli (sysadmin author, http://www.everythingsysadmin.com), Johnny Long (founder, Hackers for Charity, http://www.hackersforcharity.org/), Ryan Russell (author, Stealing the Network), Robert Scoble (blogger, http://scobleizer.com), and Amrit Williams (blogger, Nominations are open untilOctober 23, with contest winner and runners up announced on November 4, 2009, at the USENIX Large Installation System Administrator (LISA) Conference at the Baltimore Marriott Waterfront Hotel.

Until I read this, it never occurred to me that oceanographic vessels would have sysadmins, but now I know better. A truly fascinating entry from the log book of a marine sys admin.

My SysAdmin is a Rock Star because…: He takes on everything.  Out where this sysadmin rocks, the boundaries of responsibility blur— where need and capability converge, duty becomes defined.

Take two global class, nearly continuously operating oceanographic research vessels from the world’s premier oceanographic research institution, sprinkle in two near-coastal research ships, add several research laboratories worth of acoustic devices, chemistry sets, underwater robots, remote communication equipment, internet terminals, GPS-backed network time servers, precision laser motion sensors, a typical small-building network-infrastructure packed into standard ship pathways, magnetometers, user laptops, distributed inventory and workflow-tracking systems, gravimeters, old-school physical plotters required by important big-whig scientists, and multiple vehicle information systems, let simmer over 40+ years of continual scientific expeditions and constantly changing requirements, add multiple months of lengthy away-time at sea, sweeten with vacation in cool foreign ports and the hope (miniscule?) of rack-time with hot young grad students, and you have the recipe for one of the hardest sysadmin jobs the world has to offer.

For this rock-star, a typical day might look like this:

• 2 AM wakeup call from the bridge.  Problem starting navigation software after ancient PC got rebooted. Uh-oh—driving ship is important!  Hurry to the bridge and find a super-old NT4 windows machine with potentially lame virtually untested software from a low-volume sales hardware vendor.  Try to start software—something complains of missing .DLLs.  Perform awesome sleuthing and find the DLLs in an old partial backup. Carefully reconstruct the software installation by manually copying the dlls into place.  Encounter some kind of IRQ conflict or mouse failure which sends the input device into pure insanity mode. Fix the input device and realize that this probably happened before and resulted in haphazard deleting of files from the important software installation directory.
• Help a new crew-member and complete computer neophyte access their email.
• Notice a voltage change across one of the monitoring sensors on ancient all-analog gravity sensor. Swap out boards with available spares until fixed.
• Reboot crashed multibeam acquisition and visualization workstation.
• Explain to worried scientist that low-receive-power high-noise acoustic doppler current profiler data was likely caused by bubbles near the transducer head.
• Find a bug in Cisco IOS firewall configuration left by a ’short-timer’ predecessor.
• Manually re-target satellite antenna after losing signal when vessel turned and mast blocked the path to the satellite.
• Pore over fortran code from the early 90’s, still actively running on ancient Sun server, to figure out and document a legacy data processing task.
• Show the captain how to get music onto his new portable MP3 player.
• Eat lunch, try and fail to kill the conversation which a friendly crewmember, seeking common ground, tries to start about the good ol’ days of being a PC-head.
• Notice unrealistic values coming from sensor on underway seawater flow-through system. Track down to an overtightened valve slowing flow of water  through a source pipette.
• Explain to boss scientist that the internet is slow because of high latency, limited bandwidth, network saturation and that the only partial fix is to take internet away from some people.
• Take the internet away from some people so science mission can proceed as required. Try not to be hated in the eyes of the live-aboard workers
• Avoiding the evil eyes, eat dinner by your rock-star self.
• Remove malware and install virus scanning software on personal netbook brought onto network which has seen too many foreign port networks
• Help debug some PERL code a scientist brought aboard from a colleague to talk to their custom built data acquisition system.
• Hit the hot tub. Try to time it to provide an opportunity to meet that cute foreign grad student.
• Fall exhausted into rack.  Let the sound of water sloshing in the roll-tranks echo like applause in your ears for another hard, long day faced.

Shaped by a harsh environment, he who rises to the occasion is a true rock star.

This entry is short but gets right to the point. This Sysadmin of the Year nominee has saved his organization big bucks. That gets attention in any phase of an economic cycle. Names changed to keep the awesome out of the hands of headhunters.

My SysAdmin is a Rock Star because…: [Julien Sorel] virtually single handedly transformed the fortunes of [our] site and company – reducing out hosting costs by over 50% while at the same time improving our performance by over 4 times!

As a result of his work, hundreds of thousands of [end users] around the world can, and do, [access our services] in peace…. unconcerned by downtime or issues.

And he’s done this mostly between the hours of 8PM – 2AM, working with people from across 4 continents.

And that’s just the [our] staff

In an unconventional environment and with an unknown quantity …  he’s come to the party, and made sure that it’s rocking.

My only hesitation in nominating him, is that it might make someone else want to steal him from us

After a few tentative drips, it’s pouring Sysadmin of the Year entries. The following describes the indoor adventures of “Nathaniel ‘Natty’ Bumppo,” exorcist of entropy. No one is really named Natty Bumppo. Hollywood even changed the name for the 1985 “Last of the Mohicans,” a great disappointment to sticklers for literary accuracy. But we digress.

Of course, my vote — as always — goes to our very own [Natty Bumppo], the very epitome of everything wonderful about sysadmins. [Natty] is level-headed and calm, technically skilled, bright and quick, tireless and impassioned. He cares about the systems and he cares about the people who use them. He has beaten DoS attacks, tuned and maintained our hardware to a startling level of reliability, and has gotten out of a warm bed more times than I can count to battle demon entropy. What’s more, [Natty] cares about systems in general, with a deep commitment to justice and freedom on the network. Thank you [Natty], you’re the sysadmin of the century in my books.

“Attorneys at their worst”? The kind with scaly skins, cold blooded metabolic systems, and phobic to daylight?  (Entry redacted to maintain contest objectivity and not to embarrass anyone except lawyers.)

[Hans Castorp] deals with the worst of lawyers, who range from the computer-savvy to those who are completely technically-clueless, and with their legal assistants, all while keeping his cool, injecting his own brand of humor into all situations, and competently handling every situation that comes across his desk numerous times daily, 7 days a week, 24 hours a day, even while on vacation.  We are a large [specialist] law firm that handles the legal aspects of [client] projects worldwide as well as contractors dealing with [public sector organizations], and thus we help make a significant positive contribution to our nation’s economy.  Mr. [Castorp] is the techno-force behind the legal minds that make this happen.  Any man who can deal with attorneys like he does–attorneys at their worst–is highly deserving of the SysAdmin of the Year award!

Without prejudging the competition in any way, the following stands out as a very strong Sysadmin of the Year nomination. Hans Delbruck is not the nominee’s real name, of course, and we invite readers to send their guesses on why we used it as an alias for this philanthropist and saint among sysadmins.

My SysaAn Exdmin is a Rock Star because…

I’d like to nominate [Hans Delbruck] for Sys Admin of the Year. To start with, [Hans] is an avid learner. He completed several of his certifications on his own by reading books, articles, and forums. He also spreads his knowledge; he’s the go-to person when the junior team members need guidance or help and he’s always available to answer questions whether it is from a senior server team member or an up-and-coming help desk technician. Even though [Hans] keeps a heavy workload he continually takes the time to help mentor and make suggestions to others about their projects and tasks. [Hans] is continuously working after hours to resolve and work on business critical applications in our client environments, and he always displays an active interest in improving our clients’ IT environments and is always eager to encourage others to make suggestions for improvement.

[Hans] has a long track record of helping clients save time and money. For example, he was working on a project to reduce one company’s total cost of ownership regarding the purchases of hard disks for their SAN. This company had paid consultants to install and setup the SAN originally months prior. The client was not happy with the performance as well as the fact that they had to constantly purchase additional hard drives to improve performance. [Hans] worked diligently to get an evaluation SAN on site and set it up in a new configuration with a reduced number of disks, sufficient hard drive space to cover the client for 2-3 years growth and with increased performance of the system. The end result was [Hans] was able to save the company [a significant amount of money] in hardware and support purchases over the next 3 years.

[Hans] is a very skilled troubleshooter and often comes up with “out of the box” solutions. A memorable example occurred during a data center move for a client a few months ago.  One of the servers was badly damaged during transit and refused to boot up. Of course, it was the most important server to the client’s operations! The client desperately needed the data. Unfortunately, you can’t just plug a RAID 5 set into a different server and have it work. After an hour of fruitless troubleshooting, [Hans] decided to “Frankenstein” the server. He found another server with a similar motherboard, placed it next to the broken server, and moved the daughterboard holding the RAID Controller to the new server. With the hard drives still pulling power from the broken server but plugged into the daughterboard in the new server, [Hans] was able to boot up the new server and retrieve the data! We have a picture of this “Frankenstein” server, and it truly saved the day for our client.

In conclusion, I believe that [Hans’] drive, initiative, go-getter personality, and troubleshooting skills make him the Sys Admin of the year. At the very least, he’s my company’s Sys Admin of the year!

In response to audience requests for “More! More!” we’ve just added the “Take it to 11” blog space to the Sysadmin of the Year website. While the SAOTY competition is a sincere celebration of sysadminhood, we thought it would be fun to set up a virtual playground for the Keith Moon fans out there to unwind a bit.

“Take it to 11″ has become so ingrained in the culture it hardly seems worth explaining its provenance. Nigel Tufnel, lead guitarist for Spinal Tap–the loudest band in England–customized his amp volume control to go all the way up to 11. Let the Pages, Claptons, and Hendrixes of the world limit themselves to 10, feebly reasoned Tufnel. I’ve never been quite sure why Nige’ needed a volume control in the first place, but have reconciled it as some kind of industrial safety measure.

Can ordinary Joes (Strummer) and Janises (Joplin) post content? Yes, but you have to get past the bouncer. Throw it on the wall and see if it sticks.