Gluff - a DHCP lease logger for ISC-dhcpd

Gluff

Gluff news and discussions are now available at http://forum.liss.nu. Gluff is now also subject to continuous integration builds using Jenkins, which also results in automatic distributions being generated. You can find the source and binary archives at my Artifactory server.

What little documentation there is is available in the README file within the Gluff distribution.

AttachmentSize
DHCP test rig setup: A VMware case study534.47 KB

Comments

gluff[5800]: gluff v1.7.2 starting, using Sqlite3 database /var/db/dhcpd_queue.db3 and MySQL database mysql://dhcpd@192.168.1.101/dhcpd_leases
gluff[5800]: Failed to reset queue entries: no such table: lease_queue

This occurs on the failover dhcp
the primary writes data to the mysql db.
but the failover doesn't

here is how it looks like:
mysql> select * from leases;
+----+---------------------+---------------------+------+-----+-----+
| ip | lstart | lend | hw | cid | rid |
+----+---------------------+---------------------+------+-----+-----+
| 1 | 2009-10-30 09:50:49 | 2009-10-30 10:10:06 | 1 | 0 | 0 |
| 2 | 2009-10-30 09:52:00 | 2009-10-30 09:53:07 | 2 | 0 | 0 |
| 3 | 2009-10-30 10:09:01 | 2009-10-30 11:12:43 | 1 | 0 | 0 |
+----+---------------------+---------------------+------+-----+-----+
3 rows in set (0.00 sec)

mysql> select * from leases;
+----+---------------------+---------------------+------+-----+-----+
| ip | lstart | lend | hw | cid | rid |
+----+---------------------+---------------------+------+-----+-----+
| 1 | 2009-10-30 09:50:49 | 2009-10-30 10:10:06 | 1 | 0 | 0 |
| 2 | 2009-10-30 09:52:00 | 2009-10-30 09:53:07 | 2 | 0 | 0 |
| 3 | 2009-10-30 10:09:01 | 2009-10-30 11:14:30 | 1 | 0 | 0 |
+----+---------------------+---------------------+------+-----+-----+
3 rows in set (0.00 sec)

Any hint appreciated

Seems to work as expected, depenbding on the circumstances. Email me so we can work out what's wrong!

/Hans

I`m using gluff 1.7.1 in production system.
Yes, I know, that using patches in productions highly not recommended, but this feature is very useful :-\

True story :
Two Fedora 10 Linux Servers (Intel(R) Xeon(TM) CPU 2.80GHz, 4GB RAM, SCSI drives) acts as dhcpd server for 20000 clients
Dhcpd servers is in pool failover/balancing configurations, and only one server is patched with gluff + mysqld service.

I had no problems for a three months, but today, after software upgrade some of clients (stb amino`s, about 3000 boxes) it heppens:
1st server (that acts as Primary heartbeat and gluff server) - starts dropping udp packets... clients can`t get their addresses for a long time...
Netstat -na looks like Recv-Q buffer full:
[root@dhcp1]# netstat -na | grep 0.0.0.0:67
udp 110880 0 0.0.0.0:67 0.0.0.0:*

Dhcperf helped me understand problem:
Results of tests were sadly:
-----------------------------------------
1. Primary dhcpd, heartbeat on, gluff on, clients on:
Beginning DHCPDISCOVER load test.
secs success failure active | minimum current maximum
Initial probe complete: High-water mark is 1 clients/second.

2. Secondary dhcpd, clients on (as secondary helper):
Initial probe complete: High-water mark is 11 clients/second.
Beginning test run: 5 clients/second for 120 seconds.
Stopping run after 16 seconds; 3/62 clients failed.

3. Secondary dhcpd with turned off primary(without gluff) with ALL clients:
Initial probe complete: High-water mark is 26 clients/second.
Beginning test run: 13 clients/second for 120 seconds.
Stopping run after 8 seconds; 2/87 clients failed.

4. Primary dhcpd, heartbeat off (without clients, only failover updates), gluff on:
Initial probe complete: High-water mark is 7 clients/second.
Beginning test run: 3 clients/second for 120 seconds.
Succeeded: 0/360 clients failed.
--------------------------------------------

As we see - Subj :-(

Did you ever find a solution to this problem?

I believe I did, however it's not trivial. Since dhcpd is a single-threaded application doing a lot of different things within that single thread, there will always be a risk that the workload gets too high and incoming messages overflow the queue. This applies to other "heavy" features too, like DDNS.

One solution would be to increase the number of DHCP servers, of course. Another could be to increase the input buffers on the servers to handle higher maxima.

What I would like to do is to rewrite the dhcpd patch to introduce a second thread that handles the Gluff stuff. This is not as easy as it may sound, but I still think it may be a feasible solution. I just need to take the time to implement and test this.

/H

Please send me an email and I can work with you to try to resolve this.

/Hans

noc\at\mail\.primorye\.ru

Found an issue in gluff.c where it will not prolong a lease ever and will always create a new one in the mysql db. I've written a patch that fixes it, very simple.
Where you see the following
long thathw, thatcid, thatrid;

Change to the following:
long thathw, thatcid, thatrid;
thathw = 0;
thatcid = 0;
thatrid = 0;

These variables were not initialized and were becoming a null value that could not be changed. After initializing them they are able to be used.

Hi Hans

Did you have a patch 4.1.0 ?

By the way - would it be cleaner a easier to patch if you separated you code from the dhcp source - and just made som function calls from dhcp source ? - this would also make patching easer ? :)

/Regards
/Michael

Yes, I do have a patch. However, I'm not sure where I put it. :)

More seriously, I had a problem with the version of autoconf/automake I was using not being the same as was used originally, which made the patch unnecessarily large. I'll see if I can break out most of the custom code into a separate source file and produce a proper patch, ASAP.

I've enable contact info in these comments now so you can leave me your email address if you want to be notified. If there's enough interest I will set up a mailing list or a forum.

/Hans

how exactly are you supposed to patch dhcp with the included patch file?

anyone have the command syntax?

Unpack the correct dhcpd source archive, cd to the new directory and patch with the "patch" command, using the -p flag to strip off one level from the directory paths in the patch file:
Something like this:

tar xzvf dhcp-4.1.0a1.tar.gz 
cd dhcp-4.1.0a1
patch -p1 < ../dhcp-4.1.0a1-ldb.patch

By the way, I actually have a version of the patch for dhcp-4.1.0 (not a1) but haven't had time to publish it here yet.

/Hans

Hans

thanks for the reply. any idea on how i can get my hands on version dhcp-4.1.0a1?

Also, when trying to compile the GLUFF on ES4 i get the following errors:

[root@SecondaryDHCP gluff-1.6]# make
gcc -I. -g -O2 -I/usr/local/include -Wall -DHAVE_CONFIG_H -DPRODUCT=\"gluff\" -D VERSION=\"1.6\" -c -o gluff.o gluff.c
gluff.c: In function `get_id':
gluff.c:60: error: `MYSQL_STMT' undeclared (first use in this function)
gluff.c:60: error: (Each undeclared identifier is reported only once
gluff.c:60: error: for each function it appears in.)
gluff.c:60: error: `stmt' undeclared (first use in this function)
gluff.c:61: error: `MYSQL_BIND' undeclared (first use in this function)
gluff.c:61: error: syntax error before "param"
gluff.c:64: warning: implicit declaration of function `mysql_stmt_init'
gluff.c:69: warning: implicit declaration of function `mysql_stmt_prepare'
gluff.c:74: error: `param' undeclared (first use in this function)
gluff.c:76: error: `MYSQL_TYPE_STRING' undeclared (first use in this function)
gluff.c:83: warning: implicit declaration of function `mysql_stmt_bind_param'
gluff.c:88: error: `result' undeclared (first use in this function)
gluff.c:90: error: `MYSQL_TYPE_LONG' undeclared (first use in this function)
gluff.c:95: warning: implicit declaration of function `mysql_stmt_bind_result'
gluff.c:100: warning: implicit declaration of function `mysql_stmt_execute'
gluff.c:105: warning: implicit declaration of function `mysql_stmt_store_result'
gluff.c:110: warning: implicit declaration of function `mysql_stmt_num_rows'
gluff.c:111: warning: implicit declaration of function `mysql_stmt_fetch'
gluff.c:112: warning: implicit declaration of function `mysql_stmt_free_result'
gluff.c:113: warning: implicit declaration of function `mysql_stmt_close'
gluff.c:148: warning: implicit declaration of function `mysql_stmt_insert_id'
gluff.c: In function `do_find_lease':

Have you installed MYSQL on your server ? - it seems that the compiler can't find mysql headers

I utilized XAMPP which installs MYSQL, but in a different folder.

Perhaps there is a way to tell the compiler where it is?

Even if you have installed the MySQL programs, you may still be missing the client libs. You need to figure out how to get libmysqlclient.a and all the mysql header files installed. They often seem to come in a separate package.

The configure script *should* fail if the client libs are actually missing, though. If you mail me your config.log, Makefile and the full output of "make" to the email address in the README file, I can take a look and see if I can figure it out.

Oh, and do a "find / -name libmysqlclient.a" too and send me the output.

/H

Hans,

Thank you very much for the assistance. I am not sure where to send the files, as i don't see an email address on your site. Should i just upload the output as a post?

The email address is at the bottom of the README file included in the "gluff" archive.

/Hans

sorry for all the posts.. but..

I have both DHCP and mysql running on the same server. The ./configure script completes with no errors, but I am not able to compile..

Actually those three are used only as return parameters from a single function, and the function gets references (pointers) to the three variables, and completely ignores their values.

Still, it might be possible that you can get weird results if the function fails for some reason and doesn't change their values, in which case the result is unpredictable if the variables are uninitialized. I've added code to initialize them to -1 now. New official version sometime soon.