Dirty pages linux что это

Настройка параметров ядра Linux для оптимизации PostgreSQL

Оптимальная производительность PostgreSQL зависит от правильно определенных параметров операционной системы. Плохо настроенные параметры ядра ОС могут привести к снижению производительности сервера базы данных. Поэтому обязательно, чтобы эти параметры были настроены в соответствии с сервером базы данных и его рабочей нагрузкой. В этом посте мы обсудим некоторые важные параметры ядра Linux, которые могут повлиять на производительность сервера базы данных и способы их настройки.

SHMMAX / SHMALL

SHMMAX — это параметр ядра, используемый для определения максимального размера одного сегмента разделяемой памяти (shared memory), который может выделить процесс Linux. До версии 9.2 PostgreSQL использовал System V (SysV), для которой требуется настройка SHMMAX. После 9.2 PostgreSQL переключился на разделяемую память POSIX. Так что теперь требуется меньше байтов разделяемой памяти System V.

До версии 9.3 SHMMAX был наиболее важным параметром ядра. Значение SHMMAX задается в байтах.

Аналогично, SHMALL — это еще один параметр ядра, используемый для определения
общесистемного объема страниц разделяемой памяти (shared memory). Чтобы просмотреть текущие значения SHMMAX, SHMALL или SHMMIN, используйте команду ipcs.

SHM* Details — Linux

SHM* Details — MacOS X

PostgreSQL использует System V IPC для выделения разделяемой памяти. Этот параметр является одним из наиболее важных параметров ядра. Всякий раз, когда вы получаете следующие сообщения об ошибках, это означает, что у вас более старая версия PostgreSQL и у вас очень низкое значение SHMMAX. Ожидается, что пользователи будут корректировать и увеличивать значение в соответствии с разделяемой памятью, которую они собираются использовать.

Возможные ошибки неправильной конфигурации

Если SHMMAX настроен неправильно, вы можете получить ошибку при попытке инициализировать кластер PostgreSQL с помощью команды initdb.

initdb Failure
DETAIL: Failed system call was shmget(key=1, size=2072576, 03600).

HINT: This error usually means that PostgreSQL’s request for a shared memory segment exceeded your kernel’s SHMMAX parameter.
You can either reduce the request size or reconfigure the kernel with larger SHMMAX. To reduce the request size (currently 2072576 bytes),
reduce PostgreSQL’s shared memory usage, perhaps by reducing shared_buffers or max_connections.

If the request size is already small, it’s possible that it is less than your kernel’s SHMMIN parameter,
in which case raising the request size or reconfiguring SHMMIN is called for.

The PostgreSQL documentation contains more information about shared memory configuration. child process exited with exit code 1

Аналогично, вы можете получить ошибку при запуске сервера PostgreSQL используя команду pg_ctl.

pg_ctl Failure
DETAIL: Failed system call was shmget(key=5432001, size=14385152, 03600).

HINT: This error usually means that PostgreSQL’s request for a shared memory segment exceeded your kernel’s SHMMAX parameter.

You can either reduce the request size or reconfigure the kernel with larger SHMMAX.; To reduce the request size (currently 14385152 bytes), reduce PostgreSQL’s shared memory usage, perhaps by reducing shared_buffers or max_connections.

If the request size is already small, it’s possible that it is less than your kernel’s SHMMIN parameter,
in which case raising the request size or reconfiguring SHMMIN is called for.

The PostgreSQL documentation contains more information about shared memory configuration.

Понимание различий в определениях

Определение параметров SHMMAX/SHMALL немного отличается в Linux и MacOS X:

• Linux: kernel.shmmax, kernel.shmall
• MacOS X: kern.sysv.shmmax, kern.sysv.shmall

Изменение параметров ядра на MacOS X

Изменение параметров ядра на Linux

_{Не забудьте: чтобы сделать изменения постоянными, добавьте эти значения в /etc/sysctl.conf}

Большие страницы (Huge Pages)

В Linux по умолчанию используются страницы памяти 4 КБ, в BSD — Super Pages, а в Windows — Large Pages. Страница — это часть оперативной памяти, выделенная процессу. Процесс может иметь несколько страниц в зависимости от требований к памяти. Чем больше памяти требуется процессу, тем больше страниц ему выделено. ОС поддерживает таблицу выделения страниц для процессов. Чем меньше размер страницы, тем больше таблица, тем больше времени требуется для поиска страницы в этой таблице страниц. Поэтому большие страницы позволяют использовать большой объем памяти с уменьшенными накладными расходами; меньше просмотров страниц, меньше ошибок страниц, более быстрые операции чтения/записи через большие буферы. Как результат — улучшение производительности.

PostgreSQL поддерживает большие страницы только в Linux. По умолчанию Linux использует 4 КБ страниц памяти, поэтому в случаях, когда операций с памятью слишком много, необходимо устанавливать страницы большего размера. Наблюдается прирост производительности при использовании больших страниц размером 2 МБ и до 1 ГБ. Размер большой страницы может быть установлен во время загрузки. Вы можете легко проверить параметры большой страницы и их использование на вашем Linux-компьютере, используя команду cat /proc/meminfo | grep -i huge.

Получение информации о больших страницах (только на Linux)

В этом примере, хотя размер большой страницы установлен в 2048 (2 МБ), общее количество больших страниц имеет значение 0. Это означает, что большие страницы отключены.

Скрипт определения количества больших страниц

Это простой скрипт возвращает необходимое количество больших страниц. Запустите скрипт на вашем сервере Linux, пока работает PostgreSQL. Убедитесь, что для переменной среды $PGDATA задан каталог данных PostgreSQL.

Получение цифры требуемых больших страниц

Вывод скрипта выглядит следующим образом:

Вывод скрипта

Рекомендуемое значение больших страниц — 88, поэтому вы должны установить значение 88.

Установка больших страниц

Проверьте большие страницы сейчас, вы увидите, что большие страницы не используются (HugePages_Free = HugePages_Total).

Снова информация о больших страницах (только на Linux)

Теперь задайте параметр huge_pages «on» в $PGDATA/postgresql.conf и перезапустите сервер.

И снова информация о больших страницах (только на Linux)

Теперь вы можете видеть, что используются очень мало больших страниц. Давайте теперь попробуем добавить некоторые данные в базу данных.

Некоторые операции с базой данных для утилизации больших страниц

Давайте посмотрим, используем ли мы сейчас больше больших страниц, чем раньше.

Еще раз информация о больших страницах (только на Linux)

Теперь вы можете видеть, что большинство больших страниц используется.

_{Примечание: примерное значение для HugePages, используемое здесь, очень низкое, что не является нормальным значением для машины на продуктовой среде. Пожалуйста, оцените необходимое количество страниц для вашей системы и установите их соответственно в зависимости от нагрузки и ресурсов.}

vm.swappiness

vm.swappiness — это еще один параметр ядра, который может влиять на производительность базы данных. Этот параметр используется для управления поведением подкачки (swappiness) (подкачки страниц в память и из нее) в Linux. Значение варьируется от 0 до 100. Он определяет, сколько памяти будет выгружено или выгружено. Ноль означает отключение обмена, а 100 означает агрессивный обмен.

Вы можете получить хорошую производительность, установив более низкие значения.

Установка значения 0 в более новых ядрах может привести к тому, что OOM Killer (процесс очистки памяти в Linux) убьет процесс. Таким образом, можно безопасно установить значение 1, если хотите минимизировать подкачку. Значение по умолчанию в Linux — 60. Более высокое значение заставляет MMU (блок управления памятью) использовать больше пространства подкачки, чем ОЗУ, тогда как более низкое значение сохраняет больше данных/кода в памяти.

Меньшее значение — хорошая ставка на улучшение производительности в PostgreSQL.

What Are Dirty Pages in Linux

Question: What are dirty pages and what is their purpose?

Whenever application/database process needs to add virtual page into physical memory but no free physical pages are left OS must clear-out remaining old pages.

Now if old page had not been written at all then this one does not need to be saved it can be simply recovered from the the data file. But if old page has been modified already then it must be preserved somewhere so application/database can re-used later on – this is called dirty page.

OS stores such dirty pages in swap files ( so it can be removed from physical memory so another ‘new’ page can be stored in physical memory )If lots of data will be removed from page cache to dirty page area – this might cause significant IO bottleneck if actual swap device is located on local disk ( sda ) and more-over cause further issues if local disk is used as well by local root ( OS ) disk.

Page cache in Linux is just a disk cache which brings additional performance to OS which helps with intensive high read/writes on files.

As ‘sub’ product of page cache is dirty page – which was explained in above example case. Dirty pages can be also observed whenever application will write to file or create file – first write will happen in page cache area – hence creating a file which 10MB file can be really fast:

Its because that file is created in memory region not actual disk – hence response time is really fast. Under the OS such thing will be noted in /proc/meminfo and more over in ‘Dirty:

Before above command will get executed – note-down the /proc/meminfo and ‘Dirty’ row:

After command is executed:

Periodically OS or application/database will initiate sync which will write actual testfile.txt to disk:

Now Oracle Database for example does not allow to do such writes into memory region as if OS will crash or if SAN LUn will fail – data will be compromised. That’s why Oracle Database requires data to be ‘in-sync’ hence all writes needs to be confirmed by backend like disk/lun before database will throw more write requests.

Normally Databases/Application periodically drop cache hence dirty pages are written to disk in small chunks. In some cases dirty pages can grow in size as maybe application/database did not configured page cache mechanism properly.

So dirty pages can write to swap files ( Swap area ) but also to special region in disk ( LUN/file-system ). If for example we create more than 100MB swap file which will be re-used later from swap file we might cause uncecessary IO issues on swap device. Enterprise systems store swap files and swap area on OS under solid state drives ( SSD ) or dedicated LUN hence local disk performance won’t be impacted ( as normally swap region is created on Local disk )

In some cases application/database might have issues internally and dirty pages will be written as swap files but will be never re-used this will cause swap area to grow and cause uncessary IOs on local disk and lead to large swap usage under OS.

To find out at what stage OS will try to dump dirty pages back to disk layer please check official kernel documentation around Virtual Memory here and look for settings like:

Above settings needs to be tuned per Database/Application requirement as OS does not have any ‘best practice’ setting for them – they are tuned per DB/APP load/configuration.

Whenever application/database will demand memory pages to be free on physical memory – OS tends to keep everything in page cache – hence OS will need to re-allocate some of the pages and mark them as dirty. This process is works fine if application/database end are properly tuned and scaled – otherwise it will cause really aggressive swappiness to occur – as OS will need to write all dirty pages back to swap disk – this can be controlled via vm.swappiness setting.

If application/database will do agreessive swappiness it might cause serious IO writes on swap device and lead to serious system stalls – always make sure that application/databases are properly configured in terms of memory management.

As explained not all pages will be marked as dirty – mostly unused pages will get discarded rather than marked as dirty ( it all depends if pages which already are allocated were modified or not )

To verify which PIDs are using swap area – bellow command can be used:

Releasing ‘consumed’ swap space is really limited, normally if PID exits properly or simply gets shutdown swap space will be re-claimed but killing PID or if it ends-up abnormally like segfault might still leave swap space consumed. Another option is to reboot as doing swapoff and swapon command can cause serious issues or even lead to system panic state.

Documentation for /proc/sys/vm/¶

For general info and legal blurb, please look in Documentation for /proc/sys .

This file contains the documentation for the sysctl files in /proc/sys/vm and is valid for Linux kernel version 2.6.29.

The files in this directory can be used to tune the operation of the virtual memory (VM) subsystem of the Linux kernel and the writeout of dirty data to disk.

Default values and initialization routines for most of these files can be found in mm/swap.c.

Currently, these files are in /proc/sys/vm:

nr_trim_pages (only if CONFIG_MMU=n)

admin_reserve_kbytes¶

The amount of free memory in the system that should be reserved for users with the capability cap_sys_admin.

admin_reserve_kbytes defaults to min(3% of free pages, 8MB)

That should provide enough for the admin to log in and kill a process, if necessary, under the default overcommit ‘guess’ mode.

Systems running under overcommit ‘never’ should increase this to account for the full Virtual Memory Size of programs used to recover. Otherwise, root may not be able to log in to recover the system.

How do you calculate a minimum useful reserve?

sshd or login + bash (or some other shell) + top (or ps, kill, etc.)

For overcommit ‘guess’, we can sum resident set sizes (RSS). On x86_64 this is about 8MB.

For overcommit ‘never’, we can take the max of their virtual sizes (VSZ) and add the sum of their RSS. On x86_64 this is about 128MB.

Changing this takes effect whenever an application requests memory.

compact_memory¶

Available only when CONFIG_COMPACTION is set. When 1 is written to the file, all zones are compacted such that free memory is available in contiguous blocks where possible. This can be important for example in the allocation of huge pages although processes will also directly compact memory as required.

compaction_proactiveness¶

This tunable takes a value in the range [0, 100] with a default value of 20. This tunable determines how aggressively compaction is done in the background. Write of a non zero value to this tunable will immediately trigger the proactive compaction. Setting it to 0 disables proactive compaction.

Note that compaction has a non-trivial system-wide impact as pages belonging to different processes are moved around, which could also lead to latency spikes in unsuspecting applications. The kernel employs various heuristics to avoid wasting CPU cycles if it detects that proactive compaction is not being effective.

Be careful when setting it to extreme values like 100, as that may cause excessive background compaction activity.

compact_unevictable_allowed¶

Available only when CONFIG_COMPACTION is set. When set to 1, compaction is allowed to examine the unevictable lru (mlocked pages) for pages to compact. This should be used on systems where stalls for minor page faults are an acceptable trade for large contiguous free memory. Set to 0 to prevent compaction from moving pages that are unevictable. Default value is 1. On CONFIG_PREEMPT_RT the default value is 0 in order to avoid a page fault, due to compaction, which would block the task from becoming active until the fault is resolved.

dirty_background_bytes¶

Contains the amount of dirty memory at which the background kernel flusher threads will start writeback.

dirty_background_bytes is the counterpart of dirty_background_ratio. Only one of them may be specified at a time. When one sysctl is written it is immediately taken into account to evaluate the dirty memory limits and the other appears as 0 when read.

dirty_background_ratio¶

Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which the background kernel flusher threads will start writing out dirty data.

The total available memory is not equal to total system memory.

dirty_bytes¶

Contains the amount of dirty memory at which a process generating disk writes will itself start writeback.

Note: dirty_bytes is the counterpart of dirty_ratio. Only one of them may be specified at a time. When one sysctl is written it is immediately taken into account to evaluate the dirty memory limits and the other appears as 0 when read.

Note: the minimum value allowed for dirty_bytes is two pages (in bytes); any value lower than this limit will be ignored and the old configuration will be retained.

dirty_expire_centisecs¶

This tunable is used to define when dirty data is old enough to be eligible for writeout by the kernel flusher threads. It is expressed in 100’ths of a second. Data which has been dirty in-memory for longer than this interval will be written out next time a flusher thread wakes up.

dirty_ratio¶

Contains, as a percentage of total available memory that contains free pages and reclaimable pages, the number of pages at which a process which is generating disk writes will itself start writing out dirty data.

The total available memory is not equal to total system memory.

dirtytime_expire_seconds¶

When a lazytime inode is constantly having its pages dirtied, the inode with an updated timestamp will never get chance to be written out. And, if the only thing that has happened on the file system is a dirtytime inode caused by an atime update, a worker will be scheduled to make sure that inode eventually gets pushed out to disk. This tunable is used to define when dirty inode is old enough to be eligible for writeback by the kernel flusher threads. And, it is also used as the interval to wakeup dirtytime_writeback thread.

dirty_writeback_centisecs¶

The kernel flusher threads will periodically wake up and write old data out to disk. This tunable expresses the interval between those wakeups, in 100’ths of a second.

Setting this to zero disables periodic writeback altogether.

drop_caches¶

Writing to this will cause the kernel to drop clean caches, as well as reclaimable slab objects like dentries and inodes. Once dropped, their memory becomes free.

To free pagecache:

To free reclaimable slab objects (includes dentries and inodes):

To free slab objects and pagecache:

This is a non-destructive operation and will not free any dirty objects. To increase the number of objects freed by this operation, the user may run sync prior to writing to /proc/sys/vm/drop_caches. This will minimize the number of dirty objects on the system and create more candidates to be dropped.

This file is not a means to control the growth of the various kernel caches (inodes, dentries, pagecache, etc…) These objects are automatically reclaimed by the kernel when memory is needed elsewhere on the system.

Use of this file can cause performance problems. Since it discards cached objects, it may cost a significant amount of I/O and CPU to recreate the dropped objects, especially if they were under heavy use. Because of this, use outside of a testing or debugging environment is not recommended.

You may see informational messages in your kernel log when this file is used:

These are informational only. They do not mean that anything is wrong with your system. To disable them, echo 4 (bit 2) into drop_caches.

extfrag_threshold¶

This parameter affects whether the kernel will compact memory or direct reclaim to satisfy a high-order allocation. The extfrag/extfrag_index file in debugfs shows what the fragmentation index for each order is in each zone in the system. Values tending towards 0 imply allocations would fail due to lack of memory, values towards 1000 imply failures are due to fragmentation and -1 implies that the allocation will succeed as long as watermarks are met.

The kernel will not compact memory in a zone if the fragmentation index is <= extfrag_threshold. The default value is 500.

highmem_is_dirtyable¶

Available only for systems with CONFIG_HIGHMEM enabled (32b systems).

This parameter controls whether the high memory is considered for dirty writers throttling. This is not the case by default which means that only the amount of memory directly visible/usable by the kernel can be dirtied. As a result, on systems with a large amount of memory and lowmem basically depleted writers might be throttled too early and streaming writes can get very slow.

Changing the value to non zero would allow more memory to be dirtied and thus allow writers to write more data which can be flushed to the storage more effectively. Note this also comes with a risk of pre-mature OOM killer because some writers (e.g. direct block device writes) can only use the low memory and they can fill it up with dirty data without any throttling.

hugetlb_shm_group¶

hugetlb_shm_group contains group id that is allowed to create SysV shared memory segment using hugetlb page.

laptop_mode¶

laptop_mode is a knob that controls “laptop mode”. All the things that are controlled by this knob are discussed in How to conserve battery power using laptop-mode .

legacy_va_layout¶

If non-zero, this sysctl disables the new 32-bit mmap layout — the kernel will use the legacy (2.4) layout for all processes.

lowmem_reserve_ratio¶

For some specialised workloads on highmem machines it is dangerous for the kernel to allow process memory to be allocated from the “lowmem” zone. This is because that memory could then be pinned via the mlock() system call, or by unavailability of swapspace.

And on large highmem machines this lack of reclaimable lowmem memory can be fatal.

So the Linux page allocator has a mechanism which prevents allocations which could use highmem from using too much lowmem. This means that a certain amount of lowmem is defended from the possibility of being captured into pinned user memory.

(The same argument applies to the old 16 megabyte ISA DMA region. This mechanism will also defend that region from allocations which could use highmem or lowmem).

The lowmem_reserve_ratio tunable determines how aggressive the kernel is in defending these lower zones.

If you have a machine which uses highmem or ISA DMA and your applications are using mlock(), or if you are running with no swap then you probably should change the lowmem_reserve_ratio setting.

The lowmem_reserve_ratio is an array. You can see them by reading this file:

But, these values are not used directly. The kernel calculates # of protection pages for each zones from them. These are shown as array of protection pages in /proc/zoneinfo like followings. (This is an example of x86-64 box). Each zone has an array of protection pages like this:

These protections are added to score to judge whether this zone should be used for page allocation or should be reclaimed.

In this example, if normal pages (index=2) are required to this DMA zone and watermark[WMARK_HIGH] is used for watermark, the kernel judges this zone should not be used because pages_free(1355) is smaller than watermark + protection[2] (4 + 2004 = 2008). If this protection value is 0, this zone would be used for normal page requirement. If requirement is DMA zone(index=0), protection[0] (=0) is used.

zone[i]’s protection[j] is calculated by following expression:

The default values of lowmem_reserve_ratio[i] are

256	(if zone[i] means DMA or DMA32 zone)
32	(others)

As above expression, they are reciprocal number of ratio. 256 means 1/256. # of protection pages becomes about “0.39%” of total managed pages of higher zones on the node.

If you would like to protect more pages, smaller values are effective. The minimum value is 1 (1/1 -> 100%). The value less than 1 completely disables protection of the pages.

max_map_count:¶

This file contains the maximum number of memory map areas a process may have. Memory map areas are used as a side-effect of calling malloc, directly by mmap, mprotect, and madvise, and also when loading shared libraries.

While most applications need less than a thousand maps, certain programs, particularly malloc debuggers, may consume lots of them, e.g., up to one or two maps per allocation.

The default value is 65530.

memory_failure_early_kill:¶

Control how to kill processes when uncorrected memory error (typically a 2bit error in a memory module) is detected in the background by hardware that cannot be handled by the kernel. In some cases (like the page still having a valid copy on disk) the kernel will handle the failure transparently without affecting any applications. But if there is no other uptodate copy of the data it will kill to prevent any data corruptions from propagating.

1: Kill all processes that have the corrupted and not reloadable page mapped as soon as the corruption is detected. Note this is not supported for a few types of pages, like kernel internally allocated data or the swap cache, but works for the majority of user pages.

0: Only unmap the corrupted page from all processes and only kill a process who tries to access it.

The kill is done using a catchable SIGBUS with BUS_MCEERR_AO, so processes can handle this if they want to.

This is only active on architectures/platforms with advanced machine check handling and depends on the hardware capabilities.

Applications can override this setting individually with the PR_MCE_KILL prctl

memory_failure_recovery¶

Enable memory failure recovery (when supported by the platform)

1: Attempt recovery.

0: Always panic on a memory failure.

min_free_kbytes¶

This is used to force the Linux VM to keep a minimum number of kilobytes free. The VM uses this number to compute a watermark[WMARK_MIN] value for each lowmem zone in the system. Each lowmem zone gets a number of reserved free pages based proportionally on its size.

Some minimal amount of memory is needed to satisfy PF_MEMALLOC allocations; if you set this to lower than 1024KB, your system will become subtly broken, and prone to deadlock under high loads.

Setting this too high will OOM your machine instantly.

min_slab_ratio¶

This is available only on NUMA kernels.

A percentage of the total pages in each zone. On Zone reclaim (fallback from the local zone occurs) slabs will be reclaimed if more than this percentage of pages in a zone are reclaimable slab pages. This insures that the slab growth stays under control even in NUMA systems that rarely perform global reclaim.

The default is 5 percent.

Note that slab reclaim is triggered in a per zone / node fashion. The process of reclaiming slab memory is currently not node specific and may not be fast.

min_unmapped_ratio¶

This is available only on NUMA kernels.

This is a percentage of the total pages in each zone. Zone reclaim will only occur if more than this percentage of pages are in a state that zone_reclaim_mode allows to be reclaimed.

If zone_reclaim_mode has the value 4 OR’d, then the percentage is compared against all file-backed unmapped pages including swapcache pages and tmpfs files. Otherwise, only unmapped pages backed by normal files but not tmpfs files and similar are considered.

The default is 1 percent.

mmap_min_addr¶

This file indicates the amount of address space which a user process will be restricted from mmapping. Since kernel null dereference bugs could accidentally operate based on the information in the first couple of pages of memory userspace processes should not be allowed to write to them. By default this value is set to 0 and no protections will be enforced by the security module. Setting this value to something like 64k will allow the vast majority of applications to work correctly and provide defense in depth against future potential kernel bugs.

mmap_rnd_bits¶

This value can be used to select the number of bits to use to determine the random offset to the base address of vma regions resulting from mmap allocations on architectures which support tuning address space randomization. This value will be bounded by the architecture’s minimum and maximum supported values.

This value can be changed after boot using the /proc/sys/vm/mmap_rnd_bits tunable

mmap_rnd_compat_bits¶

This value can be used to select the number of bits to use to determine the random offset to the base address of vma regions resulting from mmap allocations for applications run in compatibility mode on architectures which support tuning address space randomization. This value will be bounded by the architecture’s minimum and maximum supported values.

This value can be changed after boot using the /proc/sys/vm/mmap_rnd_compat_bits tunable

nr_hugepages¶

Change the minimum size of the hugepage pool.

hugetlb_optimize_vmemmap¶

This knob is not available when the size of ‘struct page’ (a structure defined in include/linux/mm_types.h) is not power of two (an unusual system config could result in this).

Enable (set to 1) or disable (set to 0) HugeTLB Vmemmap Optimization (HVO).

Once enabled, the vmemmap pages of subsequent allocation of HugeTLB pages from buddy allocator will be optimized (7 pages per 2MB HugeTLB page and 4095 pages per 1GB HugeTLB page), whereas already allocated HugeTLB pages will not be optimized. When those optimized HugeTLB pages are freed from the HugeTLB pool to the buddy allocator, the vmemmap pages representing that range needs to be remapped again and the vmemmap pages discarded earlier need to be rellocated again. If your use case is that HugeTLB pages are allocated ‘on the fly’ (e.g. never explicitly allocating HugeTLB pages with ‘nr_hugepages’ but only set ‘nr_overcommit_hugepages’, those overcommitted HugeTLB pages are allocated ‘on the fly’) instead of being pulled from the HugeTLB pool, you should weigh the benefits of memory savings against the more overhead (

2x slower than before) of allocation or freeing HugeTLB pages between the HugeTLB pool and the buddy allocator. Another behavior to note is that if the system is under heavy memory pressure, it could prevent the user from freeing HugeTLB pages from the HugeTLB pool to the buddy allocator since the allocation of vmemmap pages could be failed, you have to retry later if your system encounter this situation.

Once disabled, the vmemmap pages of subsequent allocation of HugeTLB pages from buddy allocator will not be optimized meaning the extra overhead at allocation time from buddy allocator disappears, whereas already optimized HugeTLB pages will not be affected. If you want to make sure there are no optimized HugeTLB pages, you can set “nr_hugepages” to 0 first and then disable this. Note that writing 0 to nr_hugepages will make any “in use” HugeTLB pages become surplus pages. So, those surplus pages are still optimized until they are no longer in use. You would need to wait for those surplus pages to be released before there are no optimized pages in the system.

nr_hugepages_mempolicy¶

Change the size of the hugepage pool at run-time on a specific set of NUMA nodes.

nr_overcommit_hugepages¶

Change the maximum size of the hugepage pool. The maximum is nr_hugepages + nr_overcommit_hugepages.

nr_trim_pages¶

This is available only on NOMMU kernels.

This value adjusts the excess page trimming behaviour of power-of-2 aligned NOMMU mmap allocations.

A value of 0 disables trimming of allocations entirely, while a value of 1 trims excess pages aggressively. Any value >= 1 acts as the watermark where trimming of allocations is initiated.

The default value is 1.

numa_zonelist_order¶

This sysctl is only for NUMA and it is deprecated. Anything but Node order will fail!

‘where the memory is allocated from’ is controlled by zonelists.

(This documentation ignores ZONE_HIGHMEM/ZONE_DMA32 for simple explanation. you may be able to read ZONE_DMA as ZONE_DMA32…)

In non-NUMA case, a zonelist for GFP_KERNEL is ordered as following. ZONE_NORMAL -> ZONE_DMA This means that a memory allocation request for GFP_KERNEL will get memory from ZONE_DMA only when ZONE_NORMAL is not available.

In NUMA case, you can think of following 2 types of order. Assume 2 node NUMA and below is zonelist of Node(0)’s GFP_KERNEL:

Type(A) offers the best locality for processes on Node(0), but ZONE_DMA will be used before ZONE_NORMAL exhaustion. This increases possibility of out-of-memory(OOM) of ZONE_DMA because ZONE_DMA is tend to be small.

Type(B) cannot offer the best locality but is more robust against OOM of the DMA zone.

Type(A) is called as “Node” order. Type (B) is “Zone” order.

“Node order” orders the zonelists by node, then by zone within each node. Specify “[Nn]ode” for node order

“Zone Order” orders the zonelists by zone type, then by node within each zone. Specify “[Zz]one” for zone order.

Specify “[Dd]efault” to request automatic configuration.

On 32-bit, the Normal zone needs to be preserved for allocations accessible by the kernel, so “zone” order will be selected.

On 64-bit, devices that require DMA32/DMA are relatively rare, so “node” order will be selected.

Default order is recommended unless this is causing problems for your system/application.

oom_dump_tasks¶

Enables a system-wide task dump (excluding kernel threads) to be produced when the kernel performs an OOM-killing and includes such information as pid, uid, tgid, vm size, rss, pgtables_bytes, swapents, oom_score_adj score, and name. This is helpful to determine why the OOM killer was invoked, to identify the rogue task that caused it, and to determine why the OOM killer chose the task it did to kill.

If this is set to zero, this information is suppressed. On very large systems with thousands of tasks it may not be feasible to dump the memory state information for each one. Such systems should not be forced to incur a performance penalty in OOM conditions when the information may not be desired.

If this is set to non-zero, this information is shown whenever the OOM killer actually kills a memory-hogging task.

The default value is 1 (enabled).

oom_kill_allocating_task¶

This enables or disables killing the OOM-triggering task in out-of-memory situations.

If this is set to zero, the OOM killer will scan through the entire tasklist and select a task based on heuristics to kill. This normally selects a rogue memory-hogging task that frees up a large amount of memory when killed.

If this is set to non-zero, the OOM killer simply kills the task that triggered the out-of-memory condition. This avoids the expensive tasklist scan.

If panic_on_oom is selected, it takes precedence over whatever value is used in oom_kill_allocating_task.

The default value is 0.

overcommit_kbytes¶

When overcommit_memory is set to 2, the committed address space is not permitted to exceed swap plus this amount of physical RAM. See below.

Note: overcommit_kbytes is the counterpart of overcommit_ratio. Only one of them may be specified at a time. Setting one disables the other (which then appears as 0 when read).

overcommit_memory¶

This value contains a flag that enables memory overcommitment.

When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory.

When this flag is 1, the kernel pretends there is always enough memory until it actually runs out.

When this flag is 2, the kernel uses a “never overcommit” policy that attempts to prevent any overcommit of memory. Note that user_reserve_kbytes affects this policy.

This feature can be very useful because there are a lot of programs that malloc() huge amounts of memory “just-in-case” and don’t use much of it.

The default value is 0.

See Overcommit Accounting and mm/util.c::__vm_enough_memory() for more information.

overcommit_ratio¶

When overcommit_memory is set to 2, the committed address space is not permitted to exceed swap plus this percentage of physical RAM. See above.

page-cluster¶

page-cluster controls the number of pages up to which consecutive pages are read in from swap in a single attempt. This is the swap counterpart to page cache readahead. The mentioned consecutivity is not in terms of virtual/physical addresses, but consecutive on swap space — that means they were swapped out together.

It is a logarithmic value — setting it to zero means “1 page”, setting it to 1 means “2 pages”, setting it to 2 means “4 pages”, etc. Zero disables swap readahead completely.

The default value is three (eight pages at a time). There may be some small benefits in tuning this to a different value if your workload is swap-intensive.

Lower values mean lower latencies for initial faults, but at the same time extra faults and I/O delays for following faults if they would have been part of that consecutive pages readahead would have brought in.

page_lock_unfairness¶

This value determines the number of times that the page lock can be stolen from under a waiter. After the lock is stolen the number of times specified in this file (default is 5), the “fair lock handoff” semantics will apply, and the waiter will only be awakened if the lock can be taken.

panic_on_oom¶

This enables or disables panic on out-of-memory feature.

If this is set to 0, the kernel will kill some rogue process, called oom_killer. Usually, oom_killer can kill rogue processes and system will survive.

If this is set to 1, the kernel panics when out-of-memory happens. However, if a process limits using nodes by mempolicy/cpusets, and those nodes become memory exhaustion status, one process may be killed by oom-killer. No panic occurs in this case. Because other nodes’ memory may be free. This means system total status may be not fatal yet.

If this is set to 2, the kernel panics compulsorily even on the above-mentioned. Even oom happens under memory cgroup, the whole system panics.

The default value is 0.

1 and 2 are for failover of clustering. Please select either according to your policy of failover.

panic_on_oom=2+kdump gives you very strong tool to investigate why oom happens. You can get snapshot.

percpu_pagelist_high_fraction¶

This is the fraction of pages in each zone that are can be stored to per-cpu page lists. It is an upper boundary that is divided depending on the number of online CPUs. The min value for this is 8 which means that we do not allow more than 1/8th of pages in each zone to be stored on per-cpu page lists. This entry only changes the value of hot per-cpu page lists. A user can specify a number like 100 to allocate 1/100th of each zone between per-cpu lists.

The batch value of each per-cpu page list remains the same regardless of the value of the high fraction so allocation latencies are unaffected.

The initial value is zero. Kernel uses this value to set the high pcp->high mark based on the low watermark for the zone and the number of local online CPUs. If the user writes ‘0’ to this sysctl, it will revert to this default behavior.

stat_interval¶

The time interval between which vm statistics are updated. The default is 1 second.

stat_refresh¶

Any read or write (by root only) flushes all the per-cpu vm statistics into their global totals, for more accurate reports when testing e.g. cat /proc/sys/vm/stat_refresh /proc/meminfo

As a side-effect, it also checks for negative totals (elsewhere reported as 0) and “fails” with EINVAL if any are found, with a warning in dmesg. (At time of writing, a few stats are known sometimes to be found negative, with no ill effects: errors and warnings on these stats are suppressed.)

numa_stat¶

This interface allows runtime configuration of numa statistics.

When page allocation performance becomes a bottleneck and you can tolerate some possible tool breakage and decreased numa counter precision, you can do:

When page allocation performance is not a bottleneck and you want all tooling to work, you can do:

swappiness¶

This control is used to define the rough relative IO cost of swapping and filesystem paging, as a value between 0 and 200. At 100, the VM assumes equal IO cost and will thus apply memory pressure to the page cache and swap-backed pages equally; lower values signify more expensive swap IO, higher values indicates cheaper.

Keep in mind that filesystem IO patterns under memory pressure tend to be more efficient than swap’s random IO. An optimal value will require experimentation and will also be workload-dependent.

The default value is 60.

For in-memory swap, like zram or zswap, as well as hybrid setups that have swap on faster devices than the filesystem, values beyond 100 can be considered. For example, if the random IO against the swap device is on average 2x faster than IO from the filesystem, swappiness should be 133 (x + 2x = 200, 2x = 133.33).

At 0, the kernel will not initiate swap until the amount of free and file-backed pages is less than the high watermark in a zone.

unprivileged_userfaultfd¶

This flag controls the mode in which unprivileged users can use the userfaultfd system calls. Set this to 0 to restrict unprivileged users to handle page faults in user mode only. In this case, users without SYS_CAP_PTRACE must pass UFFD_USER_MODE_ONLY in order for userfaultfd to succeed. Prohibiting use of userfaultfd for handling faults from kernel mode may make certain vulnerabilities more difficult to exploit.

Set this to 1 to allow unprivileged users to use the userfaultfd system calls without any restrictions.

The default value is 0.

Another way to control permissions for userfaultfd is to use /dev/userfaultfd instead of userfaultfd(2). See Userfaultfd .

user_reserve_kbytes¶

When overcommit_memory is set to 2, “never overcommit” mode, reserve min(3% of current process size, user_reserve_kbytes) of free memory. This is intended to prevent a user from starting a single memory hogging process, such that they cannot recover (kill the hog).

user_reserve_kbytes defaults to min(3% of the current process size, 128MB).

If this is reduced to zero, then the user will be allowed to allocate all free memory with a single process, minus admin_reserve_kbytes. Any subsequent attempts to execute a command will result in “fork: Cannot allocate memory”.

Changing this takes effect whenever an application requests memory.

vfs_cache_pressure¶

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a “fair” rate with respect to pagecache and swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are.

watermark_boost_factor¶

This factor controls the level of reclaim when memory is being fragmented. It defines the percentage of the high watermark of a zone that will be reclaimed if pages of different mobility are being mixed within pageblocks. The intent is that compaction has less work to do in the future and to increase the success rate of future high-order allocations such as SLUB allocations, THP and hugetlbfs pages.

To make it sensible with respect to the watermark_scale_factor parameter, the unit is in fractions of 10,000. The default value of 15,000 means that up to 150% of the high watermark will be reclaimed in the event of a pageblock being mixed due to fragmentation. The level of reclaim is determined by the number of fragmentation events that occurred in the recent past. If this value is smaller than a pageblock then a pageblocks worth of pages will be reclaimed (e.g. 2MB on 64-bit x86). A boost factor of 0 will disable the feature.

watermark_scale_factor¶

This factor controls the aggressiveness of kswapd. It defines the amount of memory left in a node/system before kswapd is woken up and how much memory needs to be free before kswapd goes back to sleep.

The unit is in fractions of 10,000. The default value of 10 means the distances between watermarks are 0.1% of the available memory in the node/system. The maximum value is 3000, or 30% of memory.

A high rate of threads entering direct reclaim (allocstall) or kswapd going to sleep prematurely (kswapd_low_wmark_hit_quickly) can indicate that the number of free pages kswapd maintains for latency reasons is too small for the allocation bursts occurring in the system. This knob can then be used to tune kswapd aggressiveness accordingly.

zone_reclaim_mode¶

Zone_reclaim_mode allows someone to set more or less aggressive approaches to reclaim memory when a zone runs out of memory. If it is set to zero then no zone reclaim occurs. Allocations will be satisfied from other zones / nodes in the system.

This is value OR’ed together of

Zone reclaim on

Zone reclaim writes dirty pages out

Zone reclaim swaps pages

zone_reclaim_mode is disabled by default. For file servers or workloads that benefit from having their data cached, zone_reclaim_mode should be left disabled as the caching effect is likely to be more important than data locality.

Consider enabling one or more zone_reclaim mode bits if it’s known that the workload is partitioned such that each partition fits within a NUMA node and that accessing remote memory would cause a measurable performance reduction. The page allocator will take additional actions before allocating off node pages.

Allowing zone reclaim to write out pages stops processes that are writing large amounts of data from dirtying pages on other nodes. Zone reclaim will write out dirty pages if a zone fills up and so effectively throttle the process. This may decrease the performance of a single process since it cannot use all of system memory to buffer the outgoing writes anymore but it preserve the memory on other nodes so that the performance of other processes running on other nodes will not be affected.

Allowing regular swap effectively restricts allocations to the local node unless explicitly overridden by memory policies or cpuset configurations.

Tutorial: Beginners guide on Linux Memory Management

Linux memory management is a very vast topic and it is not possible to cover all the areas in single article. I will try to give you an overview on major areas and will help you understand important terminologies related to memory management in Linux.

Overview on Linux Memory Management

• The central part of the computer is CPU and RAM is the front end portal to CPU
• Everything that is going to CPU will go through RAM
• For example, if we have a process which is loading, the process will first be loading in RAM and the CPU will get process data from RAM
• But to make it faster, the CPU has level one, level two, level three cache. That is like RAM, but on the CPU
• There’s very small amounts of cache on the CPU, because it’s very expensive, and it’s also not very useful for all the instructions.
• So the process information will be copied from RAM to CPU and the CPU will build its cache.
• Here cache plays an important part of memory management on Linux as well.

• If a user is requesting information from hard disk, it is copied to RAM, and the user will be served from RAM.
• And while the information is copied from hard disk, it is placed in what you call the page cache.
• So the page cache stores recently requested data to make it faster if the same data is needed again.
• And if the user starts modifying data, it will go to RAM as well, and from RAM it will be copied to the hard disk, but that will only happen if the data has been sitting in RAM long enough.
• Data won’t be written immediately from RAM to hard disk, but to optimize write to the hard disk, Linux works with the concept of dirty cache.
• It tries to buffer as much as data in order to create an efficient write request.
• So as you can see in this small diagram, everything that happens on your computer goes through RAM.
• So using RAM on a Linux computer is essential for the well working of the Linux operating system.

Now we will discuss the individual part of Linux memory management and understand different terminologies related to this flow.

Understanding Virtual Memory

When analysing Linux memory usage, you should know how Linux uses Virtual and Resident Memory. Virtual Memory on Linux is to be taken literally: it is a non-existing amount of memory that the Linux kernel can be referred to.

Currently my RHEL 7 Linux has 128GB of Total Physical Memory

But as you see this node also has 32TB of Virtual Memory

• Now the idea is that if in Linux, when a process is loading, this process needs memory pointers, and these memory pointers don’t really have to refer to actual physical RAM, and that is why we are using virtual memory.
• Virtual Memory is used by the Linux kernel to allow programs to make a memory reservation.
• After making this reservation, no other application can reserve the same memory.
• Making the reservation is a matter of setting pointers and nothing else. It doesn’t mean that the memory reservation is also actually going to be used.
• When a program has to use the memory it has reserved, it is going to issue a malloc system call, which means that the memory is actually going to be allocated. At that moment, we’re talking about resident memory.

What is memory over-allocation (over-commitment) and OOM?

• In top command we can see the virt column and the res column.
• The VIRT is for Virtual Memory. That’s the amount of kilobytes that currently the process has allocated.
• And RES is for Resident, that is what it is really using.
• I have sorted the output to show processes with most memory consumption.

• We can see in the VIRT section that huge amount of memory is allocated to the process such as for java 22.4GB of virtual memory is allocated while only 4.8GB is used
• If you add all of these VIRT memory to one another, you are getting far beyond the total of 128GB of physical RAM that is available in this system.
• That is what we call memory over-allocation.
• To tune the behavior of over committing memory, you can write to the /proc/sys/vm/overcommit_memory parameter. This parameter can have some values.
• The default value is 0, which means that the kernel checks if it still has memory available before granting it. If that doesn’t give you the performance you need,
• The value 1, means that the system thinks there is enough memory in all cases. This is good for performance of memory-intensive tasks but may result in processes getting killed automatically.
• The value of 2, means that the kernel fails the memory request if there is not enough memory available.

Let us understand in layman’s terms

• This means that when any application or process requests for memory, kernel will always honour that request and «give«.
• Please NOTE, that kernel gives certain amount of memory but this memory is not marked as «used«.
• Instead this is considered as «virtual memory» and only when the application or process tries to write some data into the memory, kernel will mark that section of memory as «used«.
• So if suddenly one or some process would start using more resident memory (RES), the kernel needs to honour that request, and the result is that you can reach a situation where there is no more memory available.
• That is called OOM (Out Of Memory) situation when your system is running low on memory, and it is getting out of memory
• In such case the out of memory killer becomes active, and it will kill the process that least requires to have the memory, and that is kind of a random situation, so you’ll get a random process killed if your system is running out of memory.

Understanding and Monitoring Page Cache

Above we learned about Virtual Memory and how this is important for the working of Linux environment. Another item that is quite important is the Page Cache.

Buffers vs Page cache

• RAM that is NOT used to store application data is available for buffers and page cache. So, basically, page cache and buffer are anything in RAM that’s not used for anything else.
• When your system wants to do anything with data, which are written on a hard-disk, it firstly needs to read the data from the disk and store them in RAM. The memory allocated for these data is called pagecache.
• A cache is the part of the memory which transparently stores data so that future requests for that data can be served faster. This memory is utilized by the kernel to cache disk data and improve I/O performance.
• When any kind of file/data is requested then the kernel will look for a copy of the part of the file the user is acting on, and, if no such copy exists, it will allocate one new page of cache memory and fill it with the appropriate contents read out from the disk.
• Caches are considered as system memory and they are reported as freeable because they can be easily shrunk or reclaimed as a system’s workload demands.
• Buffers are the disk block representation of the data that is stored under the page caches.
• Buffers contains the metadata of the files/data which resides under the page cache.

Let us take an example to understand this clearly :
When there is a request of any data which is present in the page cache, first the kernel checks the data in the buffers which contain the metadata which points to the actual files/data contained in the page caches. Once from the metadata the actual block address of the file is known, it is picked up by the kernel for processing.

How does kernel perform disk read write operation?

We discussed this topic briefly in the starting of this article using a diagram for Linux memory management, now we will follow the flow of disk read and write operations with page cache between user (kernel) ⇔ memory ⇔ disk

Reading from disk:

• In most cases, the kernel refers to the page cache when reading from or writing to disk.
• New pages are added to the page cache to satisfy User Mode processes’s read requests.
• If the page is not already in the cache, a new entry is added to the cache and filled with the data read from the disk.
• If there is enough free memory, the page is kept in the cache for an indefinite period of time and can then be reused by other processes without accessing the disk.

Writing to disk:

• Similarly, before writing a page of data to a block device, the kernel verifies whether the corresponding page is already included in the cache;
• If not, a new entry is added to the cache and filled with the data to be written on disk.
• The I/O data transfer does not start immediately: the disk update is delayed for a few seconds, thus giving a chance to the processes to further modify the data to be written (in other words, the kernel implements deferred write operations).

Kernel read and write operations operate on main memory. Whenever any read or write operation is performed, the kernel first needs to copy the required data into memory:

Read operation:

1. go to disk and search for data
2. write the data from disk into memory
3. perform read operation

Write Operation:

1. go to disk and search for data
2. write the data from disk into memory
3. perform write operation
4. copy the modified data back to disk

What is the advantage of keeping page cache?

In other words we can also frame this question as, why we should not clear the buffers and page cache so frequently?

To demonstrate this in Linux memory management, we will take a very simple example: I will create a small file with some random text:

Next we will synchronize cached writes to persistent storage

Next we will clear the buffers and cache.
WARNING: This will clear all the existing buffers and cache from your Linux system so you must use this cautiously and should be avoided on production environment, especially with heavy I/O Operations

Now since the caches are clear, we see that it takes around 0.031 seconds to read the file from the disk

Now since we have read the file once, it will also be available in our page cache so if we try to read the same file again, it should be picked from the page cache.

As you see now the read time is hardly 0.001 second which is much faster compared to our earlier result.

Understanding Dirty Page

As we know, the kernel keeps filling the page cache with pages containing data of block devices. So whenever a process modifies some data, the corresponding page is marked as dirty -that is, its PG_dirty flag is set. The dirty writeback thresholds based on dirty_ratio and dirty_background_ratio parameters also take HugePages into account. The final threshold is calculated as a percentage of dirty-able memory (memory which can be potentially allocated for pagecache and get ditried).

A dirty page might stay in main memory until the last possible moment — that is, until system shutdown. However, pushing the delayed-write strategy to its limits has two major drawbacks:

• If a hardware or power supply failure occurs, the contents of RAM can no longer be retrieved, so many file updates that were made since the system was booted are lost.
• The size of the page cache, and hence of the RAM required to contain it, would have to be huge — at least as big as the size of the accessed block devices.

Therefore, dirty pages are flushed (written) to disk also known as dirty writeback under the following conditions:

• The page cache gets too full and more pages are needed, or the number of dirty pages becomes too large.
• Too much time has elapsed since a page has stayed dirty.

How dirty pages are flushed or written back to disk?

There are several parameters with sysctl which control when and how this disk writeback is performed.

Most significant are:

• dirty_bytes / dirty_ratio
• dirty_background_bytes / dirty_background_ratio
• dirty_writeback_centisecs
• dirty_expire_centisecs

The writeback can be generally divided into 3 stages

First Stage: Periodic

• The kernel threads responsible for flushing dirty data are periodically woken (period based on dirty_writeback_centisecs ) to flush data, which are dirty for at least or longer than dirty_expire_centisecs .

Second Stage: Background

• Once there is enough dirty data to cross threshold based on dirty_background_* parameters, the kernel will try to flush as much as possible or at least enough to get under the background threshold.
• Note that this is done asynchronously in kernel flusher threads and while creating some overhead, it doesn’t necessarily impact other application’s workflow. (hence the name: background)

Third Stage: Active

• If the threshold based on dirty_* parameters (usually reasonably higher than background threshold) is crossed, it means that applications are producing more dirty data faster than the flusher threads manage to writeback in time.
• In order to prevent running out of memory, the tasks which produce dirty data are blocked in the write() system-call (and similar), actively waiting for the data to be flushed.

Understanding Translation Lookaside Buffers (TLB)

• When accessing memory pages, the TLB is used to find where these pages as referred to in virtual memory reside.
• They can be in physical memory. If they are already in physical memory, we call it a TLB hit, and the page can be served pretty fast.
• In case of a TLB miss, a page walk is issued to find where the memory page is, and it is loaded. This is also referred to as a minor page fault.
• A major page fault is something different. That occurs if a memory page needs to be fetched from swap.
• The TLB contains administration for all memory pages, and for that reason can grow rather big.
• The entire TLB by itself is in RAM, and the most frequently used pieces can be stored in CPU cache to speed up the process of allocating the correct memory pages.

Understanding Active and Inactive Memory

• Active memory is memory that is really being used for something, and inactive memory is memory that’s just sitting there and not being used for something.
• The essence of the difference between the two of them is in case there is a memory shortage, the Linux kernel can do something with the inactive memory.
• You can easily monitor the difference between the two of them through /proc/meminfo . Here we can see six lines referring to active and inactive memory.

• Here we have a total of 12GB Active Memory, 450MB of Inactive Memory
• The /proc/meminfo also makes a difference between Active Anonymous and Active File Memory.
• Anon (anonymous) memory refers to memory that is allocated by programs.
• Anonymous File memory refers to memory that is used as cache or buffers.
• On any Linux system, these two kinds of memory can be flagged as Active or Inactive. Inactive file memory typically exists on a server that doesn’t need the RAM for anything else.
• If memory pressure arises, the kernel can clear this memory immediately to make more RAM available.
• Inactive Anon Memory is memory that has to be allocated. However, as it hasn’t been used actively, it can be moved to a slower kind of memory. That exactly is what swap is used for.
• If in swap there’s only inactive anon memory, swap helps optimizing the memory performance of a system.
• By moving out these inactive memory pages, more memory becomes available for caching, which is good for the overall performance of a server.
• Hence, if a Linux server shows some activity in swap, that is not a bad sign at all.

Different types of swapping scenarios and risks

• If swap space is used, you should also have a look at the /proc/meminfo file, to relate the use of swap to the amount of inactive anon memory pages.
• If the amount of swap that is used is larger than the amount of anon memory pages that you observe in /proc/meminfo , it means that active memory is being swapped. That is bad news for performance, and if that happens, you must install more RAM.
• If the amount of swap that is in use is smaller than the amount of inactive anon memory pages in /proc/meminfo , there’s no problem, and you’re good.
• If, however, you have more memory in swap than the amount of inactive anonymous pages, you’re probably in trouble, because active memory is being swapped. That means that there’s too much I/O traffic, which will slow down your system.

Conclusion

Lastly I hope the this article on Linux memory management and understanding it’s terminologies was helpful. In this tutorial guide we learned about different areas of Memory used by Linux kernel to enhance and optimize system performance. This is not a complete guide of Linux memory Management but this will help you kickoff and know the basics of Linux memory. So, let me know your suggestions and feedback using the comment section.

References

Didn’t find what you were looking for? Perform a quick search across GoLinuxCloud

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

For any other feedbacks or questions you can either use the comments section or contact me form.

Thank You for your support!!

6 thoughts on “Tutorial: Beginners guide on Linux Memory Management”

Your articles are best, I appreciate it.

I am glad it helped ��

I really enjoyed the article. Thanks from Poland.
During reading I have written down few question about page cache.
1. Is page cache just the rest of RAM which isn’t occupied by all processes?
2. So is it like that if some process ends, than his freed RAM-resources becomes part of page cache area, isn’t it?
3. Can page cache be made from blocks of memory which aren’t neighbors? I mean, is page cach consistent block or not?
4. I like the example which proves why paging is efficient. But I don’t understand why there was run the `sync` command.
I need to also read your next article about difference between RES and VIRT, I am getting curious what they really are.

Thanks for your feedback
1. Yes we can say that, page cache is placed in the unused sections of RAM
2. Yes this is something explained with dirty pages
3. A block device is allocated in page cache with these three parameters
– The address bdev of the “block_device” descriptor
– The logical block number of “block” — the position of the block inside the block device
– The block size “size”
So based on the address of the descriptor, the block devices are stored independent of any location constraint
4. The data from page cache is written to the disk following it’s normal cycle as explained with dirty pages but we used sync to make the write instant instead of waiting for it. The ‘sync’ command is used only if you feel that your data from memory can be lost due to some incident so you do a force write from memory to disk using ‘sync’

You can check this article to understand the difference between RES and VIRT

A helpful post, I just passed this onto a co-worker who was doing a little analysis on that. And he in fact bought me dinner because I discovered it for him. smile.. So let me reword that: Thanks for the treat! But yeah Thank you for spending the time to talk about this, I feel strongly about it and love reading more on this topic. If possible, as you become expertise, would you mind updating your blog with more information? It is extremely helpful for me. Two thumb up for this blogpost!