I notice Java applications like to use a lot of file handles. Also, busy TCP network services (udp only uses a single socket) use up lots of sockets, which are counted as a file handle. Think, a very busy Java Message Bus..... so I have no clue why the default is 1024. But if you find your applications (or system applications that are logging to syslog) tell you that you're out of file handles, you can check to see how many you have like this:
[root@elvira ~]# ulimit -n
1024
That should be more like 65535 for a system with, oh 8G or memory. So, you need to set it in a couple of spots. When a user logs into a system, the PAM system comes into play. The limits.conf file is PAM configuration file and affects user processes and shells. The other file is sysctl.conf which sets kernel values and thus affects pretty much everything else. Here's how it's done:
#!/bin/sh # # Increase file descriptor limit to 65535 cat<< EOF >> /etc/sysctl.conf # Increase file handle limit
fs.file-max = 65535 EOF cat<< EOF >> /etc/security/limits.conf * soft nofile 65535 * hard nofile 65535 EOF sysctl -p /etc/sysctl.conf
Before running that, check the files and make sure to delete any existing entries for the above, so you don't end up with duplicates.
[root@elvira~]# ulimit -n
65535
Nice post. I assume the limitation here is 16bits, as you represented your recommended adjustment to 65535, which is also of course, the limit of suitable network ports (unless you erroneously allow the use of port 0 despite the RFC denouncing it). Perhaps the original ulimit was designed for connections to well-known ports? That in addition to file handles seems to make the default values really nonsensical. Thanks for the recommendation!
ReplyDeleteThat's right, it's the highest number you can stuff to in a 16bit binary. So, you hit lots of computing limits at that number.
ReplyDeleteI've also heard that increasing your tcp read and write buffers (/proc/sys/net/ipv4/tcp_{wmem,rmem}) via sysctl is beneficial on a busy public facing server to improve TCP performance. Have any experience with this?
ReplyDeleteA few suggestions:
ReplyDelete1. I'd recommend appending to sysctl.conf and limits.conf as to not clobber existing settings.
echo fs.file-max=123456 >> /etc/sysctl.conf
In my experience, these files are parsed sequentially so for duplicate entries the last value will be the one set.
2. fs.file-max is a system-wide setting. ulimit is a per-process tree setting. Processes that were started under the old ulimit will remain under that ulimit until restarted; ex. Apache.
3. On some systems if the process starts early in the boot order, limits.conf may not apply. You can work around this by setting the desired PAM limit by adding the ulimit (or equiv) command in the init script (or equiv.)
4. FWIW, on RHEL5.5, the max for PAM file handle limit is 1048576. On RHEL5.5, fs.file-max = 324268 on a x86_64 machine by default, and can be higher.
Andrew K! Thanks buddy, great suggestions. The heredoc has been fixed, that was a typo on my part. Thanks for that. It's a very interesting point that some services started early in the boot process don't get the setting early enough. I actually think I saw that once with a JMS that was started early enough that although it was set system wide, that process would always get 1024. After a bunch of testing I did finally set it in the init script which ended up working, but I never actually figured out why that was happening. Thanks a bunch for the comments.
ReplyDelete@jed, I have read the same and may have set them on some servers in the past. I don't think I did any perf. testing so I have no idea if it did any good or not. :) I did fool around with all that stuff to avoid TCP fingerprinting some of my systems in the past too, that was really cool too.
ReplyDeletePlay online with fun88 casino online with Free Spins - Vie Casino
ReplyDeletePlay fun88 casino online with free spins 카지노 from Playtech, fun88 soikeotot the world's favourite slots provider. Discover more. 샌즈카지노 Play fun88 slots for fun