I have been accused of ignoring the fate of httperf recently. hmmm. Well, fair enough, it's certainly been a while since I committed anything to the repository.
That said, httperf has hardly left my thoughts.
I am pretty familiar with the tool at this point, and to be quite honest I am more than a little bit frustrated with it's design. Take a look for yourself! It's got a quite a few weird design concepts built into it, include c-ized objects (come now...), an Any type (oh good), it's own event loop implementation, it's own timer implementation (which at least doesn't leak memory since I re-wrote it) and it's own http implementation. It seems to come from an era before the advent of code re-use... (No offense to David Mosberger)
That said, it does implement the workload generator aspect of it's design quite elegantly. Differentiating between sessions, connections and requests is quite a useful as well.
Anyways, I have been busy recently ripping the guts out of httperf to the point where it is hardly recognizable any more.
The event loop, timer and http implementations are going away javascript:void(0)in lieu of those built into libevent. The core of httperf and the stats collection are moving to their own shared libraries, and the workload generators are each moving to distinct binary files.
Sound good?
Showing posts with label httperf. Show all posts
Showing posts with label httperf. Show all posts
Thursday, April 16, 2009
Tuesday, December 30, 2008
httperf status - 2008-12-30
I have to acknowledge that I have been dropping the ball in regards to completing the next release of httperf. While there has been a good amount of work done on this tool since the last release (0.9.0) I just haven't gotten around to wrapping it up yet.
Anways, I had better get to it now.
Check CVS for the todo list.
Other than simple procrastination, one major reason I put off development was that autoconf support for detecting the c99 constructs I was using wasn't integrated until 2.62.
Then I was going off on tangents, trying to implement my own libevent based http client library (which is definitely a useful project in my own opinion) and working to get mtp support introduced into the kernel.
Anyways, where we stand today is that the head of cvs is not terribly stable at very very high request rates. Maybe this is because httperf has never been able to achieve the rates that we are now reaching before and it has exposed some hithertofore unkown issue.
Whatever, I'll track it down.
As for the libevent migration. Well, Adrian Chadd was kind enough to do a partial port. However there is much more work to be done in this area such as migrating timers, timeouts, dns and http control into the libevent core loop.
Long term, httperf will dramatatically shrink in size. Event logic, the http client, and metrics collection can all be loaded onto libevent and some (as of yet unwritten) plugins. httperf will be a set tools implementing various workload generators and may further evolve into a workload language implementation.
Anways, I had better get to it now.
Check CVS for the todo list.
Other than simple procrastination, one major reason I put off development was that autoconf support for detecting the c99 constructs I was using wasn't integrated until 2.62.
Then I was going off on tangents, trying to implement my own libevent based http client library (which is definitely a useful project in my own opinion) and working to get mtp support introduced into the kernel.
Anyways, where we stand today is that the head of cvs is not terribly stable at very very high request rates. Maybe this is because httperf has never been able to achieve the rates that we are now reaching before and it has exposed some hithertofore unkown issue.
Whatever, I'll track it down.
As for the libevent migration. Well, Adrian Chadd was kind enough to do a partial port. However there is much more work to be done in this area such as migrating timers, timeouts, dns and http control into the libevent core loop.
Long term, httperf will dramatatically shrink in size. Event logic, the http client, and metrics collection can all be loaded onto libevent and some (as of yet unwritten) plugins. httperf will be a set tools implementing various workload generators and may further evolve into a workload language implementation.
Friday, October 3, 2008
New version of httperf coming soon (httperf 0.9.1)!
In spite of the fact that I barely spend any time these days working on httperf, I am pleased to note that activity has not been totally absent.
Pretty quickly we'll be releasing a new version of httperf with a number of enhancements.
The most dramatic change came from Adrian Chad, who implemented a basic port to the libevent notification system. This has dramatically improved the performance of the tool (like in the realm of "orders of magnitude").
New period variation option --period=vT1,D1,T2,D2....Tn,Dn
The idleconn tool is now optional (disabled by default), and can be built using the --enable-idleconn configure option
New options to print result data in new formats (CSV and Key=Value), however this is still being tested.
The idleconn program has been re-written to use libevents non-blocking dns and connection API.
Also, a number of memory leaks have been plugged with a re-write of the timer system.
libevent is now a build requirement.
httperf will try to compile itself with the system compilers c99 option if available.
Pretty quickly we'll be releasing a new version of httperf with a number of enhancements.
The most dramatic change came from Adrian Chad, who implemented a basic port to the libevent notification system. This has dramatically improved the performance of the tool (like in the realm of "orders of magnitude").
New period variation option --period=vT1,D1,T2,D2....Tn,Dn
The idleconn tool is now optional (disabled by default), and can be built using the --enable-idleconn configure option
New options to print result data in new formats (CSV and Key=Value), however this is still being tested.
The idleconn program has been re-written to use libevents non-blocking dns and connection API.
Also, a number of memory leaks have been plugged with a re-write of the timer system.
libevent is now a build requirement.
httperf will try to compile itself with the system compilers c99 option if available.
Monday, July 14, 2008
httperf and libevent and the future of everything
Firstly, congratulations to Adrian Chadd for starting the httperf port to libevent. Already we have seen some massive performance enhancements. CPU usage on my opensuse 11.0 workstation has improved about %3000 percent.
Certainly there is some stability work to be done. I am currently getting segmentation faults with very high request loads.
The current implementation does not use a persistent event loop, which means that the other useful features that libevent provides are not available (timers for example).
Something I want to see developed going forward is an asynchronous http client library around. The http work already done with libevent is certainly a start to this, but is far from complete. Essentially, what I want is a client api where I can perform some http client action asynchronously without needing to deal with any messy details.
The intention here is to foster a deal of code-reuse between http implementation projects that all use libevent at the core. After that, portability sits firmly on libevent's shoulders (where it belongs).
Certainly there is some stability work to be done. I am currently getting segmentation faults with very high request loads.
The current implementation does not use a persistent event loop, which means that the other useful features that libevent provides are not available (timers for example).
Something I want to see developed going forward is an asynchronous http client library around. The http work already done with libevent is certainly a start to this, but is far from complete. Essentially, what I want is a client api where I can perform some http client action asynchronously without needing to deal with any messy details.
The intention here is to foster a deal of code-reuse between http implementation projects that all use libevent at the core. After that, portability sits firmly on libevent's shoulders (where it belongs).
Subscribe to:
Posts (Atom)