Using lgrp_root()

The lgrp_root() function returns the root lgroup ID.

#include <sys/lgrp_user.h>
lgrp_id_t lgrp_root(lgrp_cookie_t cookie);

The lgrp_root() function takes a cookie representing a snapshot of the lgroup hierarchy and returns the root lgroup ID.

Using lgrp_parents()

The lgrp_parents() function takes a cookie representing a snapshot of the lgroup hierarchy and returns the number of parent lgroups for the specified lgroup.

#include <sys/lgrp_user.h>
int lgrp_parents(lgrp_cookie_t cookie, lgrp_id_t child,
                 lgrp_id_t *lgrp_array, uint_t lgrp_array_size);

If lgrp_array() is not NULL and the value of lgrp_array_size is not zero, the lgrp_parents() function fills the array with parent lgroup IDs until the array is full or all parent lgroup IDs are in the array. The root lgroup has zero parents. When the lgrp_parents() function is called for the root lgroup, lgrp_array will not be filled in.

The lgrp_parents() function returns EINVAL when the cookie is invalid. The lgrp_parents() function returns ESRCH when the specified lgroup ID is not found.

Using lgrp_children()

The lgrp_children() function takes a cookie representing the calling thread's snapshot of the lgroup hierarchy and returns the number of child lgroups for the specified lgroup.

#include <sys/lgrp_user.h>
int lgrp_children(lgrp_cookie_t cookie, lgrp_id_t parent,
                  lgrp_id_t *lgrp_array, uint_t lgrp_array_size);

If lgrp_array is not NULL and the value of lgrp_array_size is not zero, the lgrp_children() function fills the array with child lgroup IDs until the array is full or all child lgroup IDs are in the array.

The lgrp_children() function returns EINVAL when the cookie is invalid. The lgrp_children() function returns ESRCH when the specified lgroup ID is not found.

Locality Group Contents

The following APIs retrieve information about the contents of a given lgroup.

Using lgrp_cpus()

The lgrp_cpus() function takes a cookie representing a snapshot of the lgroup hierarchy and returns the number of CPUs in a given lgroup.

#include <sys/lgrp_user.h>
int lgrp_cpus(lgrp_cookie_t cookie, lgrp_id_t lgrp, processorid_t *cpuids,
              uint_t count, int content);

If the cpuid[] argument is not NULL and the CPU count is not zero, the lgrp_cpus() function fills the array with CPU IDs until the array is full or all the CPU IDs are in the array.

The content argument can have the following two values:

The lgrp_cpus() function returns EINVAL when the cookie, lgroup ID, or one of the flags is not valid. The lgrp_cpus() function returns ESRCH when the specified lgroup ID is not found.

Using lgrp_mem_size()

The lgrp_mem_size() function takes a cookie representing a snapshot of the lgroup hierarchy and returns the size of installed or free memory in the given lgroup. The lgrp_mem_size() function reports memory sizes in bytes.

#include <sys/lgrp_user.h>
lgrp_mem_size_t lgrp_mem_size(lgrp_cookie_t cookie, lgrp_id_t lgrp,
                              int type, int content)

The type argument can have the following two values:

The content argument can have the following two values:

The lgrp_mem_size() function returns EINVAL when the cookie, lgroup ID, or one of the flags is not valid. The lgrp_mem_size() function returns ESRCH when the specified lgroup ID is not found.

Locality Group Characteristics

The following API retrieves information about the characteristics of a given lgroup.

Using lgrp_latency()

The lgrp_latency() function returns the latency between a CPU in one lgroup to the memory in another lgroup.

#include <sys/lgrp_user.h>
int lgrp_latency(lgrp_id_t from, lgrp_id_t to);

The lgrp_latency() function returns a value that represents the latency between a CPU in the lgroup given by the value of the from argument and the memory in the lgroup given by the value of the to argument. If both arguments point to the same lgroup, the lgrp_latency() function returns the latency value within that lgroup.

Note - The latency value returned by the lgrp_latency() function is defined by the operating system and is platform-specific. This value does not necessarily represent the actual latency between hardware devices and may only be used for comparison within one domain.

The lgrp_latency() function returns EINVAL when the lgroup ID is not valid. When the lgrp_latency() function does not find the specified lgroup ID, the 'from' lgroup does not contain any CPUs, or the 'to' lgroup does not have any memory, the lgrp_latency() function returns ESRCH.

Locality Groups and Thread and Memory Placement

This section discusses the APIs used to discover and affect thread and memory placement with respect to lgroups. The lgrp_home() function is used to discover thread placement. The meminfo(2) system call is used to discover memory placement. The MADV_ACCESS flags to the madvise(3C) function are used to affect memory allocation among lgroups. The lgrp_affinity_set() function can affect thread and memory placement by setting a thread's affinity for a given lgroup. The affinities of an lgroup may specify an order of preference for lgroups from which to allocate resources. The kernel needs information about the likely pattern of an application's memory use in order to allocate memory resources efficiently. The madvise() function, and its shared object analogue madv.so.1, provide this information to the kernel. A running process can gather memory usage information about itself by using the meminfo() system call.

Using lgrp_home()

The lgrp_home() function returns the home lgroup for the specified process or thread.

#include <sys/lgrp_user.h>
lgrp_id_t lgrp_home(idtype_t idtype, id_t id);

The lgrp_home() function returns EINVAL when the ID type is not valid. The lgrp_home() function returns EPERM when the effective user of the calling process is not the superuser and the calling process' real or effective user ID does not match the real or effective user ID of one of the threads. The lgrp_home() function returns ESRCH when the specified process or thread is not found.

Using madvise()

The madvise() function advises the kernel that a region of user virtual memory in the range starting at the address specified in addr and with length equal to the value of the len parameter is expected to follow a particular pattern of use. The kernel uses this information to optimize the procedure for manipulating and maintaining the resources associated with the specified range. Use of the madvise() function can increase system performance when used by programs that have specific knowledge of their access patterns over memory.

#include <sys/types.h>
#include <sys/mman.h>
int madvise(caddr_t addr, size_t len, int advice);

The madvise() function provides the following flags to affect how a thread's memory is allocated among lgroups:

The madvise() function returns EAGAIN when some or all of the mappings in the specified address range, from addr to addr+len, are locked for I/O. The madvise() function returns EINVAL when the value of the addr parameter is not a multiple of the page size as returned by sysconf(3C). The madvise() function returns EINVAL when the length of the specified address range is less than or equal to zero. The madvise() function returns EINVAL when the advice is invalid. The madvise() function returns EIO when an I/O error occurs while reading from or writing to the file system. The madvise() function returns ENOMEM when addresses in the specified address range are outside the valid range for the address space of a process, or the addresses in the specified address range specifiy one or more pages that are not mapped. The madvise() function returns ESTALE when the NFS file handle is stale.

Using madv.so.1

The madv.so.1 shared object enables the selective configuration of virtual memory advice for launched processes and their descendants. To use the shared object, the following string must be present in the environment:

LD_PRELOAD=$LD_PRELOAD:madv.so.1

The madv.so.1 shared object applies memory advice as specified by the value of the MADV environment variable. The MADV environment variable specifies the virtual memory advice to use for all heap, shared memory, and mmap regions in the process address space. This advice is applied to all created processes. The following values of the MADV environment variable affect resource allocation among lgroups: