• Join FlightAware (Why Join?)
  • Login
  • US Flag 
11:04PM EDT


 

Airport Tracker/Info


-or-


 

About FlightAwareFreeBSD High Capacity Server Performance Tuning

FlightAware Open Source Software

FlightAware operates many FreeBSD severs and includes some useful tuning tips for other FreeBSD operators that need a high capacity configuration. Most operating system and server settings are extremely low and will cause failures if they're not raised substantially. Unfortunately, it is often difficult to find these configuration limitations until you experience a failure.

FlightAware runs the following configurations on machines ranging from 2x2GHz/64GB to 32x3GHz/384GB, so they can be dropped in to most configurations without a problem.

This page was last updated October 2013 and reflects our usage of FreeBSD 8.2.

/boot/loader.conf

Don't remove your ZFS settings that might be at the top of this file.

The kern.ipc.maxsockets and kern.ipc.maxsockbuf settings are for any large network server.

accf_http is a socket filter that buffers incoming connctions until a complete HTTP request arrives. The accf_http_load setting loads the module and prevents an Apache warning on start. FlightAware does not expose Apache to the Internet; it's proxied by Varnish. Accordingly, this setting is unnecessary for us but useful for others.

The rest of the semaphore/shared memory/swap settings are to allow PostgreSQL to have large (we use 16-72GB) shared memory segments. The postgres web site has documentation on kernel resource settings..

kern.maxusers="1024"
kern.ipc.semmns="2048"
kern.ipc.semmni="128"
kern.ipc.shmall="33554432"
kern.ipc.shmseg="1024"

kern.ipc.shmmax=137438953472

kern.ipc.maxsockets="256000"
kern.ipc.maxsockbuf="2621440"

kern.maxswzone="335544320"

accfhttp_load="YES"

hw.igb.enable_msix=0

/etc/sysctl.conf

The shm settings are for PostgreSQL (see above).

vm.pmap.shpgperproc settings are only required for FreeBSD 8 and below.

# set max shared memory segment size to 512 megs, 512 * 1024 * 1024, we need a
# lot for PostgreSQL
kern.ipc.somaxconn=32768
kern.maxfiles=400000
kern.ipc.shmmax=68719476736

# set the maximum number of pages of shared memory,
# this should be at least
# (max mem you could possible allocate / page size in bytes), i.e.
# at least shmmax / pagesize
#
kern.ipc.shmall=16777216
kern.ipc.semmap=256

#
# lock shared memory from being swapped out
#
kern.ipc.shm_use_phys=1
# http://spatula.net:8000/blog/2007/04/freebsd-network-performance-tuning.html
net.inet.tcp.recvspace=78840
net.inet.tcp.sendspace=78840
net.local.stream.sendspace=82320
net.local.stream.recvspace=82320
net.inet.tcp.local_slowstart_flightsize=10

kern.maxfilesperproc=104856
kern.threads.max_threads_per_proc=16384
kern.ipc.nmbclusters=66560

vm.pmap.shpgperproc=500

net.inet.icmp.icmplim=1000