Home » Linux, Shell, Tuning

Tuning file descriptors limits on Linux

11 November 2008 12 Comments

What is a file descriptor ?

Is a data structure what handle a real or pseudo file. Real files are filesystem files and pseudo files are the proc files, dev files etc. File descriptors are limited per process and a limit per entire system also exist. So your limits per process cannot surpass limit per entire system.

How many file descriptors can my system support ?

To see that just run the following command in bash:

cat /proc/sys/fs/file-max
202781

How can I increase the number of file descriptors on my system ?

Just run the following command:

echo 65535 > proc/sys/fs/file-max

Where 65535 is the new number of opened file descriptors. This value will persist until the next reboot. To save as persistent value in your system you need to add next value in /etc/sysctl.conf:

fs.file-max = 65535

After editing /etc/sysctl.conf just run:

systcl -p

to apply modification on your system.

How many file descriptors are being used ?

This information is stored in /proc/sys/fs/file-nr. Run following command to see the content of this file

cat /proc/sys/fs/file-nr

And you will get and output something like:

1500 210 65535

Where:

1500 is total alocated file descriptors

210 is total of free alocated file descriptors

65535 is the total limit of the system

Another method to see the number of file descriptors opened in your system is using lsof (list open files). Run the following command to get the number of file descriptors opened:

lsof | wc -l

Why we need to increase the number of file descriptors ?

Heavy loaded servers (but not only them) need that because they are opening a large number of files, sockets or other pseudo files. For each connection a server application will open a new socket and probably other files.

What is the limit of file descriptors per process ?

To see that just run the following command:

ulimit -n

and you will get the number of file descriptors per your session (process).

How cand I modify the limit of file descriptors per session ?

Edit /etc/security/limits.conf and add:

*                soft    nofile          65535
*                hard    nofile          65535

This options will increase the number of file descriptors for all users. If you want just for a specific user replace * with the username. For example if you want to increase the limit for the apache file server just replace * with www-data (Debian specific) and after that just restart the apache (on Debian run: /etc/init.d/apache2 restart)

How many file descriptors are opened on a specific process ?

For example if you want to see how many file descriptors are opened for apache process (www-data user) just run following command:

lsof | grep www-data | wc -l

and you will get the number of file descriptors for apache http server.

To see what files are opened and you only know the pid of the process just run:

lsof -p <pid>

and the result will be something like:

smbd      17533        root  cwd       DIR                8,2       608         29 /tmp
smbd      17533        root  rtd       DIR                8,2       728          2 /
smbd      17533        root  txt       REG                8,2   3936896     109985 /usr/sbin/smbd
smbd      17533        root  mem       REG                8,2     43480      37497 /lib/libnss_nis-2.7.so
smbd      17533        root  mem       REG                8,2     35632      37493 /lib/libnss_compat-2.7.so
smbd      17533        root  mem       REG                8,2     47528      37495 /lib/libnss_files-2.7.so
smbd      17533        root  mem       REG                8,2     10200      37728 /usr/lib/gconv/IBM850.so
smbd      17533        root  mem       REG                8,2     14320     131965 /usr/lib/gconv/UTF-16.so
smbd      17533        root  mem       REG                8,2     13144      15790 /lib/libgpg-error.so.0.3.0
smbd      17533        root  mem       REG                8,2    318608      30412 /lib/libgcrypt.so.11.2.3
smbd      17533        root  mem       REG                8,2     65800      48343 /usr/lib/libtasn1.so.3.0.12
smbd      17533        root  mem       REG                8,2      7344      77870 /lib/libkeyutils-1.2.so
smbd      17533        root  mem       REG                8,2     30440     517492 /usr/lib/libkrb5support.so.0.1
smbd      17533        root  mem       REG                8,2    102784      28254 /usr/lib/libsasl2.so.2.0.22
smbd      17533        root  mem       REG                8,2   1436976      37486 /lib/libc-2.7.so
smbd      17533        root  mem       REG                8,2     31992      48491 /lib/libpopt.so.0.0.0
smbd      17533        root  mem       REG                8,2     14624      37489 /lib/libdl-2.7.so
smbd      17533        root  mem       REG                8,2     93080      37492 /lib/libnsl-2.7.so
smbd      17533        root  mem       REG                8,2     80760      37501 /lib/libresolv-2.7.so
smbd      17533        root  mem       REG                8,2     27600      65429 /lib/libacl.so.1.1.0
smbd      17533        root  mem       REG                8,2     16128      65415 /lib/libattr.so.1.1.0
smbd      17533        root  mem       REG                8,2     42928      65510 /lib/libpam.so.0.81.6
smbd      17533        root  mem       REG                8,2     39120      37488 /lib/libcrypt-2.7.so
smbd      17533        root  mem       REG                8,2    526560      37490 /lib/libm-2.7.so
smbd      17533        root  mem       REG                8,2    130224      37500 /lib/libpthread-2.7.so
smbd      17533        root  mem       REG                8,2     93536      48299 /usr/lib/libz.so.1.2.3.3
smbd      17533        root  mem       REG                8,2    542040      48349 /usr/lib/libgnutls.so.13.9.1
smbd      17533        root  mem       REG                8,2    221960       7935 /usr/lib/libcups.so.2
smbd      17533        root  mem       REG                8,2     10104     106055 /lib/libcom_err.so.2.1
smbd      17533        root  mem       REG                8,2    147680     514871 /usr/lib/libk5crypto.so.3.1
smbd      17533        root  mem       REG                8,2    619336     517485 /usr/lib/libkrb5.so.3.3
smbd      17533        root  mem       REG                8,2    174984     514852 /usr/lib/libgssapi_krb5.so.2.2
smbd      17533        root  mem       REG                8,2     57064    1635793 /usr/lib/liblber-2.4.so.2.0.5
smbd      17533        root  mem       REG                8,2    274904    1635794 /usr/lib/libldap_r-2.4.so.2.0.5
smbd      17533        root  mem       REG                8,2    127480      37483 /lib/ld-2.7.so
smbd      17533        root  mem       REG                8,2      4096     128425 /var/lib/samba/share_info.tdb

12 Comments »

  • manu said:

    Missing trailing slash: echo 65535 > proc/sys/fs/file-max

    Otherwise useful text. thx.

  • admin (author) said:

    Thanks …

  • Simon said:

    Short, concise and useful blog. Thanks.

  • aylin said:

    tnkss very god….
    Thanks for the post.

  • Ravenman said:

    I’m using this command:

    [root@testserver ~]#lsof | grep username | wc -l
    4719

    But I can’t understand it, because the limit is lower than result:

    [root@testserver ~]# ulimit -a | grep open
    open files (-n) 4096

    What am I doing / reading wrong?

  • admin (author) said:

    The fd number can differ depending on what type of login you made. If you do console, su or ssh login the number of FD can be different if you don’t have pam_limits enabled on /etc/pam.d/(ssh|su|login). Second, doing grep, probably you catch another processes which in a way or other is using your username. The correct way is to use lsof -u username and not grep.

    Regards

  • Installing a Glassfish 3.1 Cluster on Debian 6.0 only from command line | Random Bugs said:

    […] Take a look at file descriptors (Tuning file descriptors limits on Linux) under Linux if you deploy this configuration on a production […]

  • Mohammed said:

    It does not work with me, I changed the values for www-data user but I do not see any change when using “su www-data” “ulimit -Hn”, value still the default 1024.

    May I did something wrong ?! any idea?

    Thank you for this Good article!

  • Mohammed said:

    What I did was :

    nano /etc/security/limits.conf

    and appended the following:

    www-data soft nofile 4028
    www-data hard nofile 10240

    the I have rebooted the server without any success!

    Thanks

  • admin (author) said:

    How do you check to see if is working ? if you are doing su – www-data the limits are not applied if you don’t uncomment

    session required pam_limits.so

    from /etc/pam.d/su

    Regards

  • Lukas Rieder said:

    Editing /etc/security/limits.conf actually only applies those limits for pam_limits.so. That module in turn only applies for _user sessions_!

    Since www-data is most likely just a regular non privileged non interactive (no sessions!) system user, editing this limits.conf file has no effect.

    If you like to set limits for login sessions, limits.conf is the way to go.

    If you like to modify the limits for daemons, you should apply those limits when starting the daemon, in the according init/upstart script.

    Best regards,

    Lukas

  • jfigueroa-vidder said:

    Thank you for a very well written post.

Leave your response!

Add your comment below, or trackback from your own site. You can also subscribe to these comments via RSS.

Be nice. Keep it clean. Stay on topic. No spam.

You can use these tags:
<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

This is a Gravatar-enabled weblog. To get your own globally-recognized-avatar, please register at Gravatar.