Recovering a Windows Server 2011 Essentials image from a USB backup drive

I have a customer which uses Windows SBS 2011 Essentials as a basic file and backup server. It runs on one of those little supermicro servers with an integrated atom chip and 4GB of RAM. Hardly a workhorse, but it is perfectly adequate for running his three person office.

Anyway, this morning I got a call saying that the server was down, and couldn't seem to be restarted.

When this kind of thing happens, I usually presume that its some kind of minor user error (like its not plugged in, or the tooth fairies pulled out the network cable overnight) and almost always, I can get in and out within a few minutes and couple magical key strokes.

Well... Today was a genuine problem requiring some actual work.

Turns out the the hard drive (yes there is just one drive in the unit) suffered some sort of mechanical fault and started clicking rather badly.

Well, a quick run to the local computer shop and a few minutes of minor surgery yielded a perfectly functional, yet completely blank file server (being blank is very bad for a server). The value the server had nothing to do with the actual cost of the equipment, but its value rests entirely on its function as a repository of all the companies data and that data is on the dead drive.

Fortunately, I had seen this coming a long time ago and the machine has been dumping daily images to an old drobo sitting on the shelf for something on the order of 3 years. Btw, those old drobo's aren't good for much, but this is one job that they do pretty well.

Anyway, recovering the image was fairly straight forward (this server doesn't have a DVD drive) so I needed to image a USB stick with the SBS 2011 Essentials installation disk, which works fine except for the fact that the Windows recovery software doesn't like magical new drives showing up when its about to recover an image, so annoyingly I needed to disconnect the USB stick just after the software had identified the recovery image and before starting the recovery process otherwise I would be whacked with error 0x80070057 which basically says that the recovery system is shaped differently than it should be.

Yawn.

All in all the recovery process is fairly straight forward, and the only thing I'd like to see added is a way to clone the recovery images to an offsite storage thingy.

Sun Blade 100 and OpenBSD

I find that I get distracted from the task at hand very easily.

My latest distraction from the FreeSWITCH on OpenBSD project has been a renewed interest in UltraSPARC hardware. I recently purchased a pair of Sun Fire V215 servers to act as routers, and on the way I got interested in the newer generations of UltraSPARC processors, which I think are very interesting for a variety of workloads that I have to deal with.

Anyway, this evening while out at my parents place I rummaged in the basement for a few minutes trying to find an old system I had left out here a few years ago.

Sure enough, sitting under a small pile of dust, I discovered my old Sun Blade 100 workstation. I had purchased the machine some years ago when I was working on a project which I wanted to ensure ran on big-endian architectures.

Plugging in the system, I was pleased to discover everything was exactly as I had left it (apparently in 2007 since it still had a OpenBSD 4.1 install disk inside it).

Quickly burning a copy of OpenBSD 5.3 for UltraSPARC, and running the install took about 10 minutes and I was pleased to discover a fully operational desktop.

OpenBSD 5.3 running out of the box on a Sun Blade 100 UltraSPARC workstation

These workstations were very useful in their day, and I am pleased to see that mine is still running.  Performance is a little lower than I would like for some activities like browsing the web, but its actually perfectly acceptable as a development environment (and nicely free of distractions too).

FreeSWITCH on OpenBSD - Project Status 2013-06-02

I've finally gotten around to making new headway with the FreeSWITCH on OpenBSD project. Although I can honestly see at least year (or more, really) of labour in front of me to produce a high quality port which I would be content to run my voice services on.

At the moment, I have a semi functional build which compiles a binary capable of executing just past the banner. I haven't checked the last call trace yet, however it looks like we are breaking just after the process forks and spins up a bunch of threads.

Recent Work

I merged a commit today that replaces the hash table wrapper that used to point at internal sqlite APIs to a the public hash table implemented by APR.  This is in reaction to an instant crash bug that I was encountering when the old system tried to access the internal sqlite malloc code.

I'll note that the original implementation of the hash table stuff used APR in the good old days (2008) and was replaced by Anthony for reasons that I am currently unclear about.

The results tested well in an impromptu set of test cases I whipped up to see if things should work, and the configurations loaded successfully, however the test of the new implementation really will only happen down the road when we start flinging calls around. 

Observations

I find the code style of the FreeSWITCH developers difficult to read for a couple reasons:

• Long symbol that_describe_what_should_be_happening_like_this
• Long prototypes that disappear off the edge of my monitor, I've seen lines with more than 180 characters which is a bit much.

I'm not going to whinge about the code style too much since it is what it is, however I've voiced my official note saying I don't like it.

Another issue I have is with the munging of dependencies. What I mean here is that the developers have taken a number of liberties in changing the internals of some dependencies. I suppose that they had their reasons, however it does make me twitch.

The more I fiddle with the internals of FreeSWITCH, the more I approve of the projects that have come from the OpenBSD guys. Especially when it comes from the perspective of portability, where the dependency trees are vanishingly small.  The FreeSWITCH perspective is somewhat along the lines of "ok, this pile of kitchen sinks seems balanced, don't fuck with it".

Current tasks

There are couple items to do right away. The first is spin up the module loading code, port some modules, and verify that the core modules are loading in what appears to be a correct manner.

Also I noticed that I am statically linking my internal libfreeswitch library and that needs to stop.

Down the road

I really dislike the manner in which freeswitch loads its configuration. On the horizon is a new launching and configuration loading change that makes me less twitchy. I'll make sure that everything I do here could be conceivably imported upstream.

Also, there are couple areas that I think may need special attention, memory handling and threading

Summary

There is lots left to do!

Running an OpenBSD Laptop

I've been a long time user of OpenBSD for various projects since I worked for Nortel back in the good old days (2004? Wow, that's nine years and counting). My experience with the operating system is mostly on howling servers that you would never run on your desk.

Anyway, I've been working on porting some software to OpenBSD for a couple months and kept thinking to myself

"Boy I should get a proper workstation setup"

At first I was just using an ssh shell to one of my servers and working off that, which was a pain in the ass when the network was slow or out, or otherwise unavailable.

Second thought was,

"Maybe a VM on my main laptop will do?"

Uh, no. I don't like VMs, and to be honest I need a bit more performance than typing into a VM console. I have friend that is a VM hipster who thinks they are the best thing since butter, but I'm just not that kind of guy.

Anyway, after setting up a workstation on a spare P4 that has been sitting on the shelf behind me for a couple years and enjoying the experience, I grabbed a Dell D620 from a local company which sold all its old gear to me for a tuppence.

Power Management

Last year I sat through a talk by Theo de Raadt where he claimed that OpenBSD has the best ACPI implementation of any of the free operating systems. After noticing that the stock installation of OpenBSD ran a little hot on the D620, I opened the acpi(4) man page which said:

"Userland may access acpi by using the apm(4) device."

After a little bit of reading, I concluded that editing rc.conf.local to start the apmd daemon (responsible for handling the power management stuff) in "keep my computer cool mode" should do the trick.

apmd_flags="-C"

On the advice of a random webpage I un-commented the line in /etc/sysctl.conf which tells the system to suspend whenever the laptop lid is closed.

machdep.lidsuspend=1

After a nice demo by Henning Braur after his talk at BSDCan this year I am pleased to confirm a nice suspend and resume experience. Not only does the laptop take a nice nap when the lid is closed, it actually gets out of bed and back to work when its open.

Docking

Along with the laptop came a dock, which I specifically wanted to enable a modest transition from mobile to fixed workstation. And after a solid 7 minutes of rummaging through man pages I couldn't find any hooks to detecting when the machine has been docked.

Basically I wanted the following to automatically happen when the laptop is docked

1. Turn off wifi
2. Disable suspend on closed lid
3. Turn on my desktop monitor
4. Turn off the LVDS laptop panel

1 and 2 require some root permissions which means configuring sudo, 3 and 4 are easy with xrandr (thanks keith packard).

Upon undocking, I want to reverse this behaviour.

So far I have two scripts sitting on my desktop called docked and undocked which I can run to manually cut things over to the preferred state. I suppose this will do for now, since I have other things that need my attention.

Does anyone know of a more elegant way to do this? Or would someone like to write up a small daemon to handle this in the background? Maybe it could be called dockd?

Summary

In anycase, getting the system up and running has not been terribly difficult, and I should have no more excuses when it comes to getting to work on the OpenBSD port of FreeSWITCH I was going on about recently.

Aastra MBU 400 and FreeSWITCH

Today I'm hanging out at my parents gas station/gallery/cafe/grocery store/internet publishing company. I took some time today to configure an Aastra MBU 400 for use with FreeSWITCH. The MBU 400 is a discontinued Aastra product apparently targeted towards residential or small office deployments where users would want portable handsets, akin to standard wireless portables that people already have in their homes.

Aastra 420d Portable handset which connects to MBU 400 base station, supports multiple handsets

The MBU 400 base station in my possession has a traditional POTS RJ11 port on the back for connecting to standard phone systems as well as a 10/100 network port to get the device onto the network and communicate with a SIP server.

It is possible to provision the network configuration for the base station through one of the handsets, selecting between static IP or DHCP. Also it seems possible to provision some aspects of the SIP configuration though the handset as well, although the experience would probably be tedious and is likely missing many fine grained settings. Like most SIP phones, the configuration can be pushed out with some proprietary gear from Aastra, which I don't own, and wouldn't buy anyway.

The MBU 400 has a web interface, for which the default credentials are:

• Username: admin
• Password: 22222

The SIP account configuration seems to require both the Registrar and Outbound Proxy fields to be filled (even if they have the same information). I pointed these towards my FreeSWITCH instance, to which I had already setup a new user extension.

Seeing as how my FreeSWITCH server is remote (I don't leave it sitting around my parents place), it needs to communicate over an IPv4 NAT.  The out of the box configuration didn't seem to work, but with a lot of diddling around with settings I managed to get it to connect.

To get inbound and outbound calling working, I needed to turn on stun, rport and turn the keepalive time down.

The MBU 400 is definitely an aged device which doesn't support a lot SIP features, notably there is no support for any type of encryption at any point in the communication process. Nor does the device support TCP SIP connections, which I prefer over UDP for a variety of reasons (which I might get into some other time). Codec support is pretty sparse and I ended up with vanilla PCMU audio.

Some conclusions:

• The device is out of production. Don't buy a new one. Support has expired from Aastra too.
• Nice for setups like my parents, it answers legacy POTS calls by default, but automatically dials out on my SIP circuit.
• Works with FreeSWITCH for inbound and outbound dialing, but I haven't tested other features like message waiting or holding calls.
• Buttons are a little small, and the ergonomics kind of suck.

Modern implementations like the Cisco SPA232D and correlated SPA302 portable handset does everything the MBU 400 does and much better.

FreeSWITCH on OpenBSD

Been a while since I posted. If you're reading this post you, you probably came in from your favourite search engine while searching for OpenBSD and FreeSWITCH.

FreeSWITCH is a software voice switch which handles any manner of voice related activities.

These days voice is pretty important, and the backend infrastructure which implements it is in a decades long process of migrating from the traditional  POTS (Plain Old Telephone System) to an all IP phone system. FreeSWITCH is definitely going to be on the platforms that ushers in the next generation of telecom. While I have some reservations about how the project is developed and maintained, it has a lot of things going for it like stability and a remarkable feature set.

Using FreesSWITCH on OpenBSD might seem like a good idea seeing as how OpenBSD has the spectacular pf firewall, excellent security history and doesn't move very far in weird and zany directions.

FreeSWITCH on the other hand can't seem to stop moving in weird and zany directions. They (even after moving to stable release tarballs) suggest that checking the source out from git is the best way to the most current and stable versions and don't blink twice at dragging the entire source tree of their dependencies into their git repository. The FreeSWITCH developers attitude towards using system versions of their dependencies ranges from aggressive no's to rampant apathy. I understand their reasoning (which is not exactly wrong), though for me this is pretty odd considering that pretty much every other major project doesn't have this problem.

So OpenBSD and FreeSWITCH... Where to start.

At the time of writing there is no port of FreeSWITCH for OpenBSD, or any substantial package built for any major open source operating system.  Nor is there likely to be without some serious effort. The only way to use the software is to jump though the git checkout hoops and run their gigantic build process through from start to finish (yawn). Debugging their build process on other operating systems than linux is a pain as well, given the dependency on stuff like gnu make, the position of the planets in the night sky and the inclusion of dependencies in their source tree.

There have been thousands of hours of effort put into making those dependencies 'work' properly on OpenBSD (nevermind all the other platforms out there). And replicating all that work into FreeSWITCH source tree just seems dumb and a waste of time (which it is).

My proposed solution (and admittedly a work in progress without a finish line in sight) is the creation something along the lines of a shallow fork of upstream FreeSWITCH that could actually be used as an OpenBSD port.

First order of business is the build system, and again since we're talking about OpenBSD, I am not talking about using autoconf, gnu make, cmake, imake, scons, or any number of build suites. Really, I'm just talking about vanilla OpenBSD make (here is an interesting thread on BSD Make). Of course, use system libraries or existing tested ports wherever possible.  The source tree should contain just those files actually needed for creating a bare installation


There are a long list of modules that also need porting, including some core requirements like mod_sofia.

After that there needs to be some work done on things like moving configuration to /etc, logs to /var/log and various other activities consistent with making FreeSWITCH a valid citizen on OpenBSD.

As I said, this is a work in progress, you can see the progress here on github.