Why is Swapping Bad for oracle?

Linux OS is a virtual memory system like any other modern operating system. The Virtual Memory Management system of Linux includes:

• Paging
• Swapping
• HugePages
• Slab allocator
• Shared memory

When almost all of the available physical memory (RAM) is started to be used in Linux, the kernel will start to swap out pages to the swap (disk space), or worse it may start swapping out entire processes. One another scenario is that it starts killing processes using the Out-of-Memory (OOM) Killer.

Swap Usage on Linux

To check swap usage on Linux you can use one of below:

• free: Seek for low (or zero) values for Swap / used:

# free -m
             total       used       free     shared    buffers     cached
Mem:          4018       3144        873          0         66       2335
-/+ buffers/cache:        742       3276
Swap:         4690          0       4690

• meminfo: Seek for SwapTotal = SwapFree

# grep Swap /proc/meminfo
SwapCached:            0 kB
SwapTotal:       4803392 kB
SwapFree:        4803380 kB

• top: Look for low (preferably zero) values of Swap / used:

# top

...
Mem:   4115320k total,  3219408k used,   895912k free,    68260k buffers
Swap:  4803392k total,       12k used,  4803380k free,  2390804k cached
...

• vmstat: Look for si / so values to be zero:

# vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0     12 871592  69308 2405188    0    0   103    36  275  500 14 13 71  1

Why is Swapping Bad?

Especially on Linux we try to avoid swapping because:

• The swapping potentially makes every memory page access a thousand (or more) times slower (and Linux swapping mechanism is not specifically fast).
• As more memory swapped, more operations take longer time
• As operations take longer time, more requests come in to be served
• The demand for resources exponentially increase

Due to scenario above, if you have a memory bound application running (like a database), if once we start swapping, most of the time there is no recovering back.

The Oracle Database SGA pages are pageable on Linux by default, and potentially those pages can be swapped out if system runs out of memory. Using HugePages  is one of the methods to make the Oracle SGA not to be swapped out at all, still you need to be careful about the configuration. To learn all about HugePages please read Document 361323.1 and references.

Conclusions

• Make sure your total SGA, PGA fit in your RAM also leaving some decent memory for process spaces and system services. See the database installation guides for more information
• Consider using HugePages on Linux
• Be very careful with your memory configuration (HugePages, Automatic Memory Management, Swap, VLM)
• Monitor your OS continuously for memory usage and swapping