The fun started after the upgrade was done and the router came back online with the upgraded IOS version. I noticed the box was acting pretty choppy and while a small amount of suck can be expected after bringing up your BGP session and slurping down the entire Internet table, this was far beyond the level of pain I was prepared for. Diagnostics were performed and that's when the real fun began, because it was at that point at which I got to see innarestin' error messages like these in the system logs:
Nov 13 00:14:10.680 CDT: %FIB-3-NOMEM: Malloc Failure, disabling CEF -Traceback= 0x606F4898 0x615CE9A0 0x615D8788 0x615D8EB0 0x615E19D8 0x615EFE08 0x60A26810 0x60A26E18 0x60A27900 0x609CD384What makes me bitter and angry about this outcome is that the box in question is $EMPLOYER's primary Internet edge router: a 7206VXR chassis populated with various PA's and an NPE-400. I shouldn't see errors about RAM but wait... how delightful! This box only has 256MB of RAM! Whereas the identically-configured standby router (that is almost never used) has the NPE's maximum of 512MB. This means my primary router can run BGP, or it can run CEF, but it can't run both -- but the unused standby can. That's splendid and is completely indicative of a well thought-out architecture for redundancy. And by that previous statement I mean is not at all any of those things.
.Nov 13 00:14:17.812 CDT: %FIB-3-NOMEM: Malloc Failure, disabling CEF -Traceback= 0x606F4898 0x615CE9A0 0x615D8788 0x615D8EB0 0x615E19D8 0x615EFE08 0x60A26810 0x60A26E18 0x60A27900 0x609CD384
.Nov 13 00:14:32.636 CDT: %SYS-2-MALLOCFAIL: Memory allocation of 4084 bytes failed from 0x60DE41EC, alignment 0
Pool: Processor Free: 25104 Cause: Memory fragmentation
Alternate Pool: None Free: 0 Cause: No Alternate pool
-Process= "BGP Router", ipl= 0, pid= 172 -Traceback= 0x606F4898 0x60851118 0x60857BE4 0x60DE41F4 0x60DE6274
Ra's perched on my desk and looking at me as if to say, "Dad, go to bed so I can curl up with you and be warm!" So I guess that's what I'll do. First thing I'll have to do when I get into the office
How’d I get so white and nerdy
