Monday, May 14, 2012

Troubleshooting the router crashing and more...

Over the past few weeks, I have been trying to find a way to optimize the speed of our router as well as finding a way such that the router does not crash when being sent a large number of packets at one time.  Once, we were able to get speeds of up to 20 Mbits/sec, but it crashed instantly after and it became frustrating from there.  A lot of it probably had to do with optimizing our code a bit as well as disabling the debug mode, but I figured the packet buffer might've been the cause of the crashing problem.  So I investigated how the buffer actually worked and ran some tests on the two buffers that were being used: one for small packets, specifically ping and ARP packets for example, and one for large packets, such as those from large file transfers in FTP or benchmark testing.  Along the way, I was able to make the buffers a lot bigger by modifying some values in ipport.h and enabled TCP window scaling to attempt to fix the router from crashing.  I had thought that by enabling TCP window scaling, packets would be able to go over the maximum window size (65,535) so the packets can handle large amounts of data at one time. However, I found out that the buffers do not actually fill up through testing and observation of the number of packets in each buffer compared to their maximum length defined within the Nichestack, so the case of buffer overflowing has been ruled out.  Matt has suggested creating a new project and make a controlled experiment to see if the router crashes with mostly original code (no changes to routing but two interfaces were enabled).  It turned out that the router still crashes with the original code. When it crashes, all the interfaces goes down, but one connection stays alive.  Matt and I suspected that another case is that somewhere in our code, the router is actually turning off the interfaces, so I am investigating that at the moment by going through the parts of the routing where the program may actually be forcing the interface off.  More to come...

No comments:

Post a Comment