This document outlines some information specific to system tuning for running all executables (nr-softmodem, lte-softmodem, nr-uesoftmodem, lte-uesoftmodem, nr-cuup, called softmodems for short in the following). Also, it explains which Linux capabilities are required to run, and how to run without sudo.

• Performance Tuning
  • CPU
  • Ethernet-based Radios
• Capabilities
  • General capabilities
  • Capabilities with DPDK
  • Capabilities with UHD
  • Capabilities with AW2S
  • Other radios

Performance Tuning

Please also refer to the advanced configuration in the tutorial, which groups many tips and tricks.

CPU

OAI used to try to set the minimum CPU-DMA latency to 2 us by writing to /dev/cpu_dma_latency. However, it is unclear if this has a significant effect. Further, in containerized workloads, it might not be possible to set this at all. We assume the user of OAI sets this value before starting OAI. See the Linux kernel documentation for more information.

OAI used to try to set the minimum frequency of cores running OAI to the maximum frequency. We assume that the user sets CPU frequency policies accordingly; in fact, tutorials generally suggest to either set a performance CPU governor, or disable any sleep states at all. Hence, setting these low-level parameters seems useless, and we assume the user of OAI disables sleep states before starting OAI.

To disable all sleep states, simply run the following:

sudo cpupower idle-set -D0

Sometimes, dependencies might not be installed, in which case you should install what cpupower asks you to install. Note that this is not persistent.

See the Linux kernel documentation for more information.

You can disable hyper-threading in the BIOS.

You can disable KPTI Protections for Spectre/Meltdown for more performance. This is a security risk. Add mitigations=off nosmt in your grub config and update grub.

Ethernet-based Radios

For ethernet-based radios, such as AW2S, some USRPs, and 7.2 radios, increase the ringbuffers to a maximum. Read on interface <fronthaul-interface-name> using option -g, then set it with -G:

ethtool -g <fronthaul-interface-name>
ethtool -G <fronthaul-interface-name> rx <maximum-rx-value> tx <maixmum-tx-value>

Also, you can increase the kernel’s default and maximum read and write socket buffer sizes to a high values, e.g., 134217728:

sudo sysctl -n -e -q -w net.core.rmem_default=134217728
sudo sysctl -n -e -q -w net.core.rmem_max=134217728
sudo sysctl -n -e -q -w net.core.wmem_default=134217728
sudo sysctl -n -e -q -w net.core.wmem_max=134217728

Capabilities

Historically, all softmodems are executed as root, typically using sudo. This remains a possibility, but we do not recommend it due to security considerations. Rather, consider giving specific capabilities as outlined below. Read capabilities(7) (man 7 capabilities) for more information on each of the below capabilities.

Note that we tested this using 5G executables; 4G should work, but have not been tested as extensively. If in doubt, run eNB/lteUE using sudo. The below comments on capabilities apply in general as well; however, 4G executable might not warn about missing capabilities or just fail.

Refer to any of the docker-compose files under ci-scripts/ to see how to give capabilities in docker. If you run from source, you can use setcap to mark a process to run with specific capabilities. For instance, you can add the “general capabilities” as described further below on the files like this:

sudo setcap cap_sys_nice+ep ./nr-softmodem
sudo setcap cap_ipc_lock+ep ./nr-softmodem
sudo setcap cap_sys_nice+ep ./nr-uesoftmodem
sudo setcap cap_ipc_lock+ep ./nr-uesoftmodem
sudo setcap cap_net_admin+ep ./nr-uesoftmodem

To make only temporary changes to capabilities, use capsh. It needs to be started with all capabilities, so needs sudo. To drop all capabilities, issue:

sudo -E capsh --drop="cap_sys_nice,cap_chown,cap_dac_read_search,cap_fowner,cap_fsetid,\
cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_linux_immutable,cap_net_bind_service,\
cap_net_broadcast,cap_net_admin,cap_net_raw,cap_ipc_lock,cap_ipc_owner,cap_sys_module,\
cap_sys_rawio,cap_sys_chroot,cap_sys_ptrace,cap_sys_pacct,cap_sys_admin,cap_sys_boot,\
cap_sys_resource,cap_sys_time,cap_sys_tty_config,cap_mknod,cap_lease,cap_audit_write,\
cap_audit_control,cap_setfcap,cap_mac_override,cap_mac_admin,cap_syslog,cap_wake_alarm,\
cap_block_suspend,cap_audit_read,cap_perfmon,cap_bpf,cap_checkpoint_restore" --print -- \
  -c "/absolute/path/to/nr-softmodem -O /absolute/path/to/config.conf"

(For readability, the command has been separated onto multiple lines through \). To run with SYS_NICE, remove the first capability (cap_sys_nice) from the list of dropped capabilities.

General capabilities

• SYS_NICE: required by all softmodems to assign a higher priority to threads to increase the likelihood that the Linux scheduler schedules a thread for CPU time. Also, in some configurations, CPU pinning is employed to ensure consistent performance, also known as setting a CPU affinity.

This capability is necessary by default when running any softmodem, for setting real-time requirements on processing threads. To allow to run without these requirements, the softmodems check if SYS_NICE is available, and skips any thread priority and affinity settings if the capability is not available. This allows to run any softmodem without root privileges in RFsim; you can see this by either observing a corresponding warning at the beginning of the execution, or the fact that no affinity/default priority is set for new threads.

• IPC_LOCK: OAI tries to lock all its virtual address space into RAM, preventing that memory from being paged to the swap area. If this capability is not present, a warning is printed.

• NET_ADMIN: Required at the UE to open the interface oaitun_ue1 or eNB/gNB in noS1 mode. 5G executables will throw an error if they cannot create or modify an interface, but will continue running. It is therefore possible to run without this capability, but you cannot inject any traffic into the system. 4G executables might need this requirement, and possibly fail.

Capabilities with DPDK

Additionally to the “general capabilities” above, you need these capabilities when running with 7.2 fronthaul, which uses the xran library with a dependency on DPDK:

• IPC_LOCK (becomes mandatory with DPDK)
• SYS_RESOURCE
• NET_RAW
• SYS_ADMIN: This is required by DPDK when using IOVA PA (Physical Address) mode to read /proc/self/pagemaps. However, if DPDK EAL is configured to use IOVA VA (Virtual Address) mode, this capability is no longer required.

Capabilities with UHD

You don’t need any additional capabilities for UHD beyond the “general capabilities” for performance outlined above. Make sure that the USB device(s) are readable and writable by the current user (e.g., uhd_usrp_probe can be executed by a non-root user).

Capabilities with AW2S

You don’t need any additional capabilities for AW2S beyond the “general capabilities” for performance outlined above.

Other radios

Other radios have not been tested. If they do not work without additional capabilities beyond the “general capabilities”, please file a bug report.