Analytics Tracking Code

Tuesday, December 30, 2008

httperf status - 2008-12-30

I have to acknowledge that I have been dropping the ball in regards to completing the next release of httperf. While there has been a good amount of work done on this tool since the last release (0.9.0) I just haven't gotten around to wrapping it up yet.

Anways, I had better get to it now.

Check CVS for the todo list.

Other than simple procrastination, one major reason I put off development was that autoconf support for detecting the c99 constructs I was using wasn't integrated until 2.62.

Then I was going off on tangents, trying to implement my own libevent based http client library (which is definitely a useful project in my own opinion) and working to get mtp support introduced into the kernel.

Anyways, where we stand today is that the head of cvs is not terribly stable at very very high request rates. Maybe this is because httperf has never been able to achieve the rates that we are now reaching before and it has exposed some hithertofore unkown issue.

Whatever, I'll track it down.

As for the libevent migration. Well, Adrian Chadd was kind enough to do a partial port. However there is much more work to be done in this area such as migrating timers, timeouts, dns and http control into the libevent core loop.

Long term, httperf will dramatatically shrink in size. Event logic, the http client, and metrics collection can all be loaded onto libevent and some (as of yet unwritten) plugins. httperf will be a set tools implementing various workload generators and may further evolve into a workload language implementation.

Monday, December 29, 2008

A spin with gnome-blog

I was poking around the software repository for opensuse 11.1, and came across a tool called gnome-blog. This is a panel applet which is supposed to allow me to post to my blog directly from the desktop rather than the supposedly tedious process of opening the blog website and posting from there.

Anyways, the tool is supposed to allow me to drag and drop images inline with the text, so I figure I will give it a whirl.

This version of gnome-blog is from 2006, and appears to be the most recent, so I suppose that development has stalled. Google has failed to deliver anything of interest more recent than that.

Update: I am not impressed. So long gnome-blog.


-Ted

Friday, December 26, 2008

Linux Kernel Space Media Transfer Protocol

So after a couple years of pondering I have finally begun work on implementing a kernel space driver for the media transfer protocol.

Yes, yes, this is my first foray since university into kernel space, but I have had enough of the user-space solutions to implementing the media transfer protocol. Hundreds of devices have now implemented the darn thing (including my iriver clix if you recall), and what's more the USB.org folks have finally published the 1.0 version of the standard.

First here is a bit of the technical reasoning.


  1. Already two distinct user-space implementations which both share similar drawbacks. libmtp, and libgphoto

  2. Dependant on libusb which has stalled in development

  3. Race condition in device connection, first application to detect and connect blocks any other software from using the device until connection is dropped



Etc, the current implementation suffers from all the problems that user space drivers suffer from in the monolithic world (even the microsoft implementation has this going on). Also, it's about time I get my hands dirty in the kernel for pete's sake.

So how is this thing going to work you might ask?

Well, here is how I see it right now, and others are welcome to chime in on this.

The protocol is implemented partly in kernel space, and partly in user-space depending on necessity and appropriateness.

Kernel Space Driver (mtp):
Connects and manages all operations:
Exposes file hierarchies as a mountable file system
Implements protocol for transferring straight files.
Issues device handles to userspace which can be used asynchronously (no more device pointers!)
Automatic device connection

User Space Library (libmtp-media)
Implements protocol for managing media (including meta-data) (music, pictures, video, etc)

User Space Library (libmtp-cal)
Implements protocol for managing contact and calendar (vcard, ical, etc)

Essentially the protocol implementation will be separated across a variety of locations.

Anywho, after an hour or so of fiddling, here is the latest line from dmesg:


usb 4-5: new high speed USB device using ehci_hcd and address 9
usb 4-5: configuration #1 chosen from 1 choice
mtp: USB Skeleton device now attached to USBSkel-192
usb 4-5: New USB device found, idVendor=4102, idProduct=112a
usb 4-5: New USB device strings: Mfr=1, Product=2, SerialNumber=3
usb 4-5: Product: iriver clix
usb 4-5: Manufacturer: iriver Limited
usb 4-5: SerialNumber: 41dff81c000000f33230303530313031
usb 4-5: USB disconnect, address 9
mtp: USB Skeleton #192 now disconnected


Not much, but hey it's the beginning of the road here.

Lastly, if anyone wants to employ me to design this, I am certainly on the market.

-Ted Bullock

Friday, October 17, 2008

How to synchronize a Nokia 6301 in Linux - Investigation

This afternoon I received my brand new Nokia 6301 cell phone. In many respects, I really do like the thing.

This nokia smartphone has a wifi antenna which allows it to be used as a VOIP handset if you are in range of an appropriate network and also supports all the normal functionality of your typical cell phone. It also supports Bluetooth and has a mini-usb adapter.

When physically connected to a modern linux machine (in my case openSUSE 11.0), it initially identifies itself as a USB Mass Storage device and the desktop responds appropriately by mounting it like any other USB drive. However, the phone specifies a couple of other connectivity options, including "Data Storage" (USB Mass Storage), "Media & Printing" (MTP) and "PC Suite".

When you choose "Media & Printing", the device switches to MTP mode (Woohoo, go libmtp!) which libmtp supports very nicely, although after perusing the datastructure, I am not clear which folders are the "correct" folders for which type of file. I guess this would just require some testing inside of Windows to see where the nokia tools put things. That said, this also means that f-spot should work pretty much out of the box.

Anyways, the third option "PC Suite" puts the device into a state which the nokia PC Suite application can use to synchronize the phone with Microsoft Outlook or Lotus Notes. This third area is the most interesting for me at the moment. I want my Evolution Contact List and Calendar to synchronize with with the phone so that I don't need to pull out my laptop just to check if my flight to New York leaves at 10:45 or 11:45.

There do seem to be a number of tools for solving this problem, however they are not particularly explanatory, nor have I actually been able to sync my calendar, todo list or contact list to the phone in yet in Linux.

So, that pretty much summarizes my experience to date with the Nokia 6301 in Linux.

I definitely would appreciate someone pointing me to documentation on how to get my calendar out of the PC and onto the phone.

Thursday, October 9, 2008

The problem with current design software

The more experience that I have with how engineering and electrical design is done, more I am astonished at how much effort goes into problems that are fundamentally trivial.

For instance, when designing a schematic, I need to spend a considerable amount of time figuring out which terminal block contacts are available, and whether or not the final wiring layout is electrically consistent with the schematic.

I don't care about that!

No kidding.

I cannot believe that I need to do that in this day and age. I should be able to plug device one into device two and my design software should just know how to make it happen. Period.

At the end of the day I want to be able to say to my machine

"Connect these devices in this order"


I should be able to just figure out basic system layout, and all the wiring, schematics and general fiddling should be just figured out for me.

Tuesday, October 7, 2008

openSUSE 11.0 and the nouveau driver

I recently got my nVidia dual-head Quadro NVS 280 PCIe graphics card functioning in openSUSE 11.0 with the open source nouveau driver!

Originally I was inclined to use the proprietary nvidia driver since the nv driver does not support multi-head setups with my particular card. However, the legacy nvidia driver is not functioning at the moment with the current kernel iteration of openSUSE 11.0. So after a few hours of frustration, I blew away the proprietary driver and began work on installing nouveau.

The binary version that is comes packaged for the distribution won't let me start X for some reason or another, and besides, I wanted to try out the latest and greatest developments towards the driver.

Following the installation instructions was straightforward after I installed the necessary devel packages that come with the distribution. And within a few minutes I had a functioning desktop cloned across my two monitors.

The next step was enabling telling randr to spread the desktop across both monitors, by folling the randr instructions on the nouveau wiki.

Clearing out a good bit of the autogenerated cruft from the xorg.conf file provided me finally with a functioning extended desktop. All this with the open source driver, and a bit of perseverance.



Thanks to the folks on IRC! Very helpful, and best wishes!

-Ted

Friday, October 3, 2008

New version of httperf coming soon (httperf 0.9.1)!

In spite of the fact that I barely spend any time these days working on httperf, I am pleased to note that activity has not been totally absent.

Pretty quickly we'll be releasing a new version of httperf with a number of enhancements.

The most dramatic change came from Adrian Chad, who implemented a basic port to the libevent notification system. This has dramatically improved the performance of the tool (like in the realm of "orders of magnitude").

New period variation option --period=vT1,D1,T2,D2....Tn,Dn

The idleconn tool is now optional (disabled by default), and can be built using the --enable-idleconn configure option

New options to print result data in new formats (CSV and Key=Value), however this is still being tested.

The idleconn program has been re-written to use libevents non-blocking dns and connection API.

Also, a number of memory leaks have been plugged with a re-write of the timer system.

libevent is now a build requirement.

httperf will try to compile itself with the system compilers c99 option if available.

Monday, July 14, 2008

httperf and libevent and the future of everything

Firstly, congratulations to Adrian Chadd for starting the httperf port to libevent. Already we have seen some massive performance enhancements. CPU usage on my opensuse 11.0 workstation has improved about %3000 percent.

Certainly there is some stability work to be done. I am currently getting segmentation faults with very high request loads.

The current implementation does not use a persistent event loop, which means that the other useful features that libevent provides are not available (timers for example).

Something I want to see developed going forward is an asynchronous http client library around. The http work already done with libevent is certainly a start to this, but is far from complete. Essentially, what I want is a client api where I can perform some http client action asynchronously without needing to deal with any messy details.

The intention here is to foster a deal of code-reuse between http implementation projects that all use libevent at the core. After that, portability sits firmly on libevent's shoulders (where it belongs).

Sunday, July 13, 2008

Open source home renovations

On June 30, 2008 I took possession of my very first property. It's an 800 square foot, two-bedroom condo in South Calgary (Alberta).

Given that I am a massive geek, part of the renovations I am taking on in preparation for moving in, is a robust home network. This includes (at least) seven cat5e, and six rg6 cable drops. For this I have used a fancy product by Leviton. Their Structured Media center items appear to be exactly what I need to marshal my assorted cables, and send them along to their eventual end points.

This allows a lot of flexibility towards future home renovations, and lets you easily re-design the layout of the network at a future point in time. As far as I am concerned, this sort of cable organization is the best way to go.

In any case, all this work is being completed with the intention of designing an ideal open source home. My routing, media distribution, work bench, telephony and data storage system have all marshaled into a single room. The original analog telephony network is untouched in case a future owner doesn't want to use the fancy system that I have built.

Anyways, I have been doing this upgrade as part of a behind the scenes project that I have been playing with to develop a home running entirely on open source software. More on this later, expect interesting developments as the year goes on.

-Ted

Wednesday, June 4, 2008

Lotus Symphony 1 Experience

The glitzy look of the recent IBM Lotus Symphony release got my attention pretty quickly. While trying it out on a couple of tasks I rather rapidly started hitting various bugs. That said, I think the OpenOffice folks could take some hints in usability from this project.

The biggest issue that I ran into was with Bold, or Heading text not showing up on a page. Technically the the word processor thinks that the text is there because I can move my cursor through it, however it just shows up as white. Closing and re-opening the document will show the text for a few moments, but it will disappear again pretty quickly.

This is a deal breaker for me. So I won't be recommending it to anyone for the time being. Too bad really.

Friday, May 16, 2008

On the Radio

On the Radio!

Earlier this week, I was minding my own business, working away and listening to one of the periodical podcasts that pass through my filter of media to consume.

The podcast in question is LugRadio, episode 100.

At about the exact moment of 76:02 into the show, they started reading an email that I had mailed about a month ago. Promptly, the guys made fun of my name, bemoaned my own bemoaning of linux support for the iriver clix (now resolved of course), and then congratulated me on a job well done. I suppose.

Anyways, I am glad to have made a difference, been laughed at, and been mentioned on the radio. Who knew.

As a side note, a passer by just noted that I am running openSUSE on my laptop. Good for him. Wooo community.

Enough for now.

Tuesday, May 13, 2008

My initial experience with pfSense

As good as OpenBSD has been as my gateway router OS, I am pretty sure that I am going to move away from it to one of the pre-packaged routing systems.

Specifically, I am talking about pfSense. pfSense is a distribution of FreeBSD that uses the pf packet filter technology originally from OpenBSD, and actually had its origins as a fork of the m0n0wall project which uses ipfilter. Both systems are extremely powerful, and importantly, easy for me to use, and because I want to stick with pf as my underlying packet filter, I have chosen pfSense.

Installation onto my compact flash driven Compaq Evo D300 Small Form Factor, was very straightforward with the assistance of my card reader, dd, and a null-modem serial cable (after I figured out that installation is done via a serial terminal rather than the screen). Certainly, anyone ready to do an embedded installation of pfsense just with parts lying around their house deserves major geek cred points.

Right now, only 64Mb of the 2Gb flash card is partitioned for use (from the default image), which actually is not really a big deal since its a router. I'm not exactly going to mess with it very often. Maybe. I am forever tinkering with this stuff. But I think that I will leave it until the next distribution update.

Monday, May 12, 2008

Compaq Evo D300 SFF as a router

About 6 months ago I purchased a reasonably cheap second hand Compaq Evo D300 Small Form Factor PC.

Originally, the machine was intended to become a dial-up router and http proxy for my moms small gallery business in Empress, Alberta. After repeated failed attempts to get OpenBSD to actually dial into the ISP, I gave up on that train of thought for the time being and brought the machine back to Calgary.

Jumping forward a bit, I decided to start using the machine (named evo on my network) as my own gateway instead. Fine. OpenBSD installed just peachy, and configuring dhcpd and dns took a couple hours. Poof, instant router. However, one of the machines problems is that unless a keyboard is attached, it will fail to boot. Despite all my rummaging through the bios, I was unable to turn off the darned "halt keyboard missing; Press F1 to continue error".

This is a problem for server hardware which run headless (like a router should be able to), since if there is a power failure, I will need to manually hook up a monitor and keyboard, and press F1, and verify it is booting. Gah, No thanks; I would rather run a crumby linksys. Sure enough, Google came to my rescue. Apparently, the ignore missing keyboard option is available on the machine, however, both a power and bios password need to configured to make it available in the BIOS.

Silly.

That said, the machine is more than capable for a SOHO router, and now I am keeping it for myself.

Thursday, April 24, 2008

I want less hardware

I have been seriously considering reducing the number of computers in my life. In the good old days, I ran a fairly powerful Pentium 4 desktop for gaming and general computer work. The machine is still my central desktop, but I have found that I rarely use it these days. That said, I promised my brother that he can have the machine.

I've been thinking about what I want to replace the desktop with, and the more I think about it, the more certain I am that I no longer really want a desktop.

My laptop (Dell Inspiron 700m) performs the majority of the tasks that I need, has open source drivers for every piece of hardware on the machine and is also very easy to bring with me where ever I go. And from personal experience, the machine is a spectacular conference laptop. Avoiding the need synchronize the machines is very appealing.

So. Is this the way I want to go?

Maybe.

For May, my plan is to pack up the desktop, give it to my brother, and see if just using the laptop works for me. At the end of the month, I will review if this was a success or not and choose whether or not to buy a new desktop.

Monday, April 21, 2008

More bugs in openSUSE 11.0 Beta 1

Although I haven't done another major test run at openSUSE 11.0, I did have noticed that Bug 343858 (gdm packages downloaded with control-center upgrade use resolution specified for external monitor) is mostly fixed. I did hit the problem with maximized windows in metacity not filling the screen in certain circumstances. All should be well on this front.

Having xrandr extend my desktop to an external monitor is currently not possible without explicitly using the Virtual keyword in xorg.conf. I filed bug 381765, however the problem is apparently inherent with RANDR 1.2

Oh well.

I still have not grasped how the new pulseaudio stuff is supposed to let me record things from my microphone. There are too many switches to play with for my emotional well-being (or something... Anyways I haven't learned it yet).

Testing a SD card in the 700m's onboard reader was problematic. An icon for the card flickered on and off the desktop about 100 times with lots of pop up windows trying to let me know of deep seeded grief, and checking dmesg shows numerous IO errors. I have not filed a bug yet, but I will sometime this week.

Anyways, it seems that the system is starting to now formally stabilize; and while this release won't meet my every wim (working nouveau? autoconf 2.62? ifolder? bongo?), I will definitely be migrating most of my machines after the release. Unless something dramatic occurs, the media server probably won't change until support concludes from Novell for 10.3.

More later.

-Ted

Saturday, April 19, 2008

openSUSE 11.0 Beta 1 on Dell 700m

My laptop is in the process of updating to the latest Factory release of openSUSE 11.0.

Certainly I have been hitting a couple major bugs:


  • Bug 376742 - Wireless doesn't work (wpa_supplicant doesn't get run)

  • Bug 343858 - gdm packages downloaded with control-center upgrade use resolution specified for external monitor



Since Alpha 2, I have been running the weekly (zypper update -t package).

Certainly things are starting to stabilize now. For instance, I have no problem using the machine as a workstation.

I have to say that the Dell 700m laptop is well supported along with a good portion of my other extraneous hardware, the majority of the core functionality works very very well.

That said, I notice that Hibernate and Suspend/Resume used to work in SUSE Linux 10.1, it doesn't anymore. I wonder if there is a way to revert to whatever mechanism was being used before...?

Also, I am unsure if the SD Card reader works or not. By working I mean, I plug an SD card into the onboard reader and a windows pops up in GNOME and asks what it should do with it. I'll test this again tonight; I think that the last time I looked into this was back in the 10.2 days, so things may have changed since then.

Next is the modem. I have never been able to successfully establish a dial-up connection with the modem. I am really not quite certain as to what the problem is. Again, this will be something that I will test out tonight.

I have also never tested the firewire port. Does anyone have a firewire external harddrive that they want to lend me?

Hmm, the last thing is the microphone. On the 700m there is a defect in the motherboard design that creates static on the microphone port. I have tested this a couple of years ago with a Windows XP install. There is certainly some sort of configuration problem with 11.0b1 and esd which won't let me turn on gnome-reclevel so I can't test this properly at the moment.

Certainly I need to do some research into the new pulseaudio stuff.

Meh.

That's a problem for another day.

Anyways, that's my experience with openSUSE 11.0 and my laptop.

Monday, April 14, 2008

Yahoo and Microsoft? Thoughts

In reviewing some of the old yahoo/microsoft news clips of the past several months, I came across this note about yahoo shareholders suing yahoo for rejecting the initial Microsoft buyout deal.

My thought is these guys are out of there minds if they think a Microsoft acquisition of Yahoo is good for them.

I simply cannot see Microsoft being capable of stewarding a transition of this type in a way that would actually work. Nevermind the completely divergent technology base and skill base. If Microsoft feels they are not able to develop their Live platform to rule the day on it own, why should anyone trust them to steward the integration of a completely different platform?

It's not that I am against Microsoft here. AND, I don't get that they know what is missing with their own strategy to begin with.

So I wonder what the Michigan’s Wayne County Employees’ Retirement System thinks they will get out of the Yahoo/Microsoft deal.

Sunday, April 13, 2008

openSUSE 11.0 and autoconf 2.62

So autoconf 2.62 missed the boat just by a couple weeks to be included in openSUSE 11.0.

Sigh.

I have been anticipating this release for quite a while now. 2.62 has a number of enhancements that affect what I do.

openmp support, along with support additional tests for the c99 stuff are a couple changes that jump mind.

hmmm

Anyways, I guess I'll be waiting for either SLED 11 or openSUSE 11.1.

Sorry httperf, you'll have to make due with 2.61 for another release.

Sigh.

hahaha

Thoughts?

-Ted

Installing opensuse and some thoughts

So I am just wrapping up my third opensuse 10.3 installation of the evening.

Certainly the installation can take quite a while (about 2.5 hours for a GNOME desktop with along with updates).

That is of course just for the base system.

After that comes, the (inevitable?) binary nvidia driver, media codecs, dvd playback and then the various development tools that I use to do what I do (icc for example).

This entire process wraps up in about 3.5 hours, and I have a working system that does pretty much everything that I could want a system to do. Notably, this takes about 2-4 hours less to complete than a comparable installation of Windows XP.

If the US would ever get its software patent nonsense in order we could finally see quicker and easier installs, and I would feel comfortable giving a disk to my parents with the expectation that they would be able to install a working system.

For me, the biggest bug that I am running into is the lack of good built-in support for the nvidia cards that seem to proliferate my life. This is one of the reasons that I am strongly considering diving into the nouveau project.

Lastly, I just want to point out that I have absolutely no complaints with opensuse. The project has been very good to me and my needs.

Thoughts?

-Ted

nvidia quadro nvs 280 cards

So I have recently purchased a number of PNY Quadro NVS 280 (nvidia) graphics cards across a variety of bus flavors.

I have a number of thoughts regarding the cards. They support dual-DVI and are passively cooled.

While they are certainly not very performant cards, they can reliably support any activity that a typical workstation would need to do.

Besides keeping a number of the cards, for myself, I think I will forward a couple of them off to the nouveau project. At some point in the near future I would love to be able to throw one of these cards into a machine of my choosing and have a 3d capable system at my finger tips.

Lastly, since I own a number of nvidia cards anyways, I have been VERY strongly thinking of getting involved in the nouveau project myself.

I will be moving later this summer to a place with a garage that I can certainly see converting into an open source development workshop.

-Ted

Thursday, April 10, 2008

Pizza and video cards

So I am just sitting down for dinner in a local pizzeria with great expectations of enjoying a very scrumptious dinner for one.

Being a complete nerd, I naturally have my laptop with me, and have connected to the restaurants wireless connection. The fact that this situation is possible, really speaks to our ability to be communication. Like really.

Even 5 years ago, what I am doing right now, would have been relatively rare. Now, however I am only one of several people in the restaurant playing with various wireless devices.

Cool hey!

Anyways, I just left my favorite computer store here in Calgary with three rather sexy video cards.

The cards are as follows:


  • PNY Low-Profile Quadro NVS 280 AGP with DMS-59 (Dual DVI-I)

  • PNY Quadro NVS 280 PCIe with DMS-59

  • Matrox G550 with DMS-59



All the cards are passively cooled, and will find loving homes in the various re-incarnated computers that pass through my life.

Wednesday, April 9, 2008

autoconf 2.62 is now released

Autoconf 2.62 has now been released.

I have been anticipating this release for more than a year now due to its support for openmp and a number of C99 compiler check enhancements.

This will be the last autoconf release under the GPL version 2; going forward we'll be seeing GPL version 3.

Spiffy.

So what's going on with ifolder?

The ifolder project is a really interesting way of synchronizing data between users and locations. I personally used the project for over a year against SuSE Linux 10.1.

Back in 2006 I filed an enhancement request for openSUSE 10.3 to include the open source ifolder server with the distribution. There was fair degree of interest on the mailing list.

To date, there has been no action by Novell to include this software with their open source distribution. This is interesting because they wrote it in the first place. If I were a conspiracy theorist, I would think that they are holding it back from the open source community just so that their enterprise distribution (OES for example) would be more value added than their community distribution.

Lucky thing I could care less about conspiracy theories.

So Novell has not put more than a token response into this request since the original request was made. Indeed I have just spent the day on IRC (#openSUSE-factory) waiting for a response to this question.

I'll try to be on their next status meeting to pursue this further.

Drafting software in the free world

So here at the new job with Rising Edge Engineering, I have been learning all about electrical drafting, document control and the meaning of life.

One of things that I see that is missing in the open source universe is suitable open source drafting software. Inkscape, and OpenOffice.org Draw are not designed for this stuff. So is there anything out there that can be used for something resembling real drafting...?

The other thing I see is missing is some sort of document control server that OO.o, inkscape, and other document apps can plug into to deal with revisions. It would be great to set up something that sits on top of ssh that is designed to deal with the various intricacies of document management stuff.

Really I want an end-to-end enterprise document creation/storage system.

Any Takers?

Wednesday, March 19, 2008

Career's a go

So I found myself an absolutely amazing engineering job!

A couple weeks ago I had an interview with a company by the name of Rising Edge Engineering. For those of you without an electrical background of sorts, a rising edge is the part of a digital electronic signal where the voltage rises from low to high, off to on, or 0 to 1.

The interview was absolutely amazing. The way that the conversation went was something like "I don't know how to do anything that you guys do! And, I am absolutely committed to learning what it takes to do this".

Their response was something along the lines of "Ooooo! I want that!"

So here I am, employed as an Engineer in Training in the discipline of Electrical Engineering for Rising Edge Engineering.

Fun Hey!

Saturday, January 19, 2008

Job search ahoy!

So I am bringing my post-graduation sabbatical to an end and am now fully into the creation of my engineering career.

I spent the past six months since graduation working in the construction industry with an awesome residential framing company known as LPT Construction. The owner and manager has been really great to work for while I was working through school, and was totally willing to work around whatever crazy school schedule I had.

I really have learned an enormous amount about construction, management, safety and productivity from the years of part and full time work in this industry.

Nevertheless, I am now really interested in starting my software engineering career; after all, I attained my degree for a reason right!

Anyways, so I am available for hire! If you want to take a look at my resume, please have at it!

-Ted