Page 1 of 1

DHCP dies

Posted: Sun Jan 12, 2014 5:09 pm
by russellhltn
Wondering if anyone else is experiencing this problem. I've had it happen twice. The 881W simply stops issuing IP addresses, but tm.lds.org indicates that only 84 out of 107 are in use. (Peak ever is 99 out of 107.)

Running a ipconfig /renew on a machine that can't get a IP and I get the message "unable to contact your DHCP server. Request has timed out."

This is happening on multiple computers, and I still have Internet connectivity on other computers connected to the same switch. (filter.lds.or also gives me a check mark, so filtering is still working.)

Everything so far seems to point to a failure of the DHCP service, while the rest of the firewall is still working.

Re: DHCP dies

Posted: Wed Jan 15, 2014 10:39 am
by TimRiker
I've seen this happen before. A reboot resolved the issue.

Re: DHCP dies

Posted: Wed Jan 15, 2014 10:48 am
by russellhltn
I returned 6 hours later - the same machines were able to get on with no problem. I'm reluctant to do a reboot of the firewall since that will create more problems for those who do have valid leases.

I need to follow up with the connectivity people. They haven't called me back.

I'm wondering if there isn't something about the numbers that lie. The implication of 84 out of 107 is that there should be 23 IPs available. But I'm wondering if the DHCP doesn't return IPs to the pool right away and perhaps I really was out of IPs.

Re: DHCP dies

Posted: Wed Jan 15, 2014 12:21 pm
by TimRiker
dunno. The reboot might just have cleared the lease pool and "fixed" the issue by starting to hand out broken leases. Worth investigation. If the lease pool is depleted, it would be handy to see that in the status.

Re: DHCP dies

Posted: Wed Jan 15, 2014 2:56 pm
by russellhltn
I talked to connectivity. They added another IP pool since I was above the threshold for adding. Off hand they weren't able to answer my question about if the numbers accurately reflect the available IP pool for new devices. I'm betting I did exhaust the lease pool and TM's numbers just are not the right numbers for seeing that.

Time will tell if this fixes things.

Re: DHCP dies

Posted: Fri Jan 17, 2014 12:51 pm
by johnshaw
russellhltn wrote:I talked to connectivity. They added another IP pool since I was above the threshold for adding. Off hand they weren't able to answer my question about if the numbers accurately reflect the available IP pool for new devices. I'm betting I did exhaust the lease pool and TM's numbers just are not the right numbers for seeing that.

Time will tell if this fixes things.
I have definetly seen my tm report our entire user subnet full, enough that I needed to call and get another added. It wasn't ever consistently showing a certain percentage used, it showed, every Sunday it MAXED the IP's out. Not sure if this helps, but I don't think the tm values are not reporting properly, I wouldn't put it past the 881's not to be configured well, but I don't think it's a tm problem.

Re: DHCP dies

Posted: Fri Jan 17, 2014 2:49 pm
by russellhltn
I did some poking around about Cisco DHCP. I do see where there is a report for "Expired leases". There's a setting on how often expired leases are cleaned up (maximum time period is 10 minutes). But I also see a setting that determines how long after a lease expires before it will be re-issued to new devices. Apparently re-issuing leases too quickly can confuse firewalls.
johnshaw wrote:I have definetly seen my tm report our entire user subnet full, enough that I needed to call and get another added. It wasn't ever consistently showing a certain percentage used, it showed, every Sunday it MAXED the IP's out.
In my case this was during a overlap of wards. The first ward was in PH/RS while the second ward was in Sacrament meeting. So it's quite likely an old lease expired from a first ward member bugging out, but the DHCP wasn't willing to issue the IP to a new device held by the second ward.

Was yours a situation of one ward soaking up all the IPs? I could see that situation where none of the leases had lapsed, so there was nothing waiting before getting re-issued.

BTW, I was told the threshold for a IP pool increase is 85%. Since my all-time high of 99 out of 107 crossed that mark, that was their first response when I got them on the phone. I'll have to see if that does it.

Re: DHCP dies

Posted: Mon Jan 20, 2014 2:15 am
by russellhltn
Sunday turned out to be interesting. I had run into a problem last week when 84 out of 107 IPs were listed as in use. Since GSD added 61 IPs, my all-time peak went from 99 to 142. Considering this was a ordinary Sunday and the 99 included things like stake conference, I have to think that a significant number of users have been denied connectivity. (Any those complaints never made its way to me.)

This time I was able to get the FHC machines on-line, so I'm not aware of a problem, but then I experienced a problem at 79% (84/107)utilization and now I'm at 85% (142/168). Clearly, the TM numbers are not telling the whole story.

I may have to figure out a way to monitor the network DHCP requests to figure out the real numbers.

Re: DHCP dies

Posted: Mon Jan 20, 2014 12:13 pm
by johnshaw
Mine was during an overlap of wards, so there were multiple getting access. I hadn't really gone back in to see the full data, but after adding my 61 addresses I'm now peaking much higher than I expected, and concluded as you have that many were getting denied. But I couldn't see that they were being denied because there were no more ip's available in the pool. But my dhcp pool and peak numbers were the same value, and did not show a percentage unused.

Re: DHCP dies

Posted: Mon Jan 20, 2014 12:48 pm
by russellhltn
johnshaw wrote:and did not show a percentage unused.
Just to be clear, I calculated the percentage based on the numbers that were provided as a way of understanding them. Although the graph does give a visual representation (the height of the blue bar reaching the top of the blue area).