It seems that changing the settings in m0n0wall didn't help a lot. The apartment manager called me up again and said instead of 10 users calling in per hour to complain, now 10 users calls in every two hours. I did more searching and found that spontaneous reboots under load on the ALIX appears to be a known issue. Plus m0n0wall's firewall states can't be changed even if I run it on a PC, so I finally decided to switch to pfSense.
pfSense started as a fork of m0n0wall, but for some reason it doesn't have the voucher system that m0n0wall has, so I didn't want to use it at first. But I found that pfSense's 2.0-ALPHA release does have the voucher system, so I decided to give that a try.
I installed pfSense on a PC with 2 GB RAM and gave it a state table size of 200,000; and on another ALIX board with 128 MB RAM, which I left the state table size at the default of 10,000. Unfortunately, during testing, I found that for the 2.0-ALPHA version, once I enable the captive portal, it says the gateway is offline and can't connect to the Internet. So I decided to go back to a more stable version (1.2.3-RC2), and let the apartment complex stay with the user login system for now, and go for the voucher system later. I also had to flash the ALIX with BIOS 0.99h to resolve a boot problem. All this took about an hour.
So I drove out 100 miles again to replace the two ALIX m0n0wall boxes with the PC and the ALIX pfSense.
And I remembered to take photos of the whole setup this time.
My two ALIX m0n0wall boxes stuck between the two Peplink load balancers. The white boxes on the top shelf are cheapo TP-Link ADSL modems.
This is hard to see since my notebook's screen made the rest of the image underexposed. There are now three ALIX boxes, the new one with pfSense installed. The PC on the bottom shelf is the one with pfSense installed.
Success! 24 hours of uptime! Previously, I couldn't even get 10 minutes of uptime with m0n0wall during peak usage. This screenshot came from the ALIX box as the platform type is nanobsd.
The ALIX 2c0's lowly AMD 433 MHz Geode LX700 CPU and 128 MB RAM coping with nearly 40 users on three 4 Mbps ADSL lines.