5. Configuration Parameter Tuning

5.1. Introduction

This section provides a detailed explanation of the configuration parameters specific to the network/transport (L3/L4) layers that can be adjusted according to the needs of your application. Tweaking some of the configuration parameters in this section can drastically affect the performance of your mOS application.

Once a mOS application is launched, the mOS stack reads a mOS configuration file (named mos.conf by default) and initializes the underlying resources and spawns the threads on behalf of the application.

Our setup.sh script automatically creates a mos.conf configuration file for each sample application and places it in the /samples/(your app name)/config/ directory. The default values of the parameters mentioned in the file would work out-of-the-box for 1/10 Gbps networks.

5.2. mOS Configuration Parameters

The following section explains the configuration parameters. The mos.conf configuration file consists of a single parameter block named mos block. While some parameters are mandatory (required for running a mOS application), others are optional.

5.2.1. mos Parameter Block

A mos parameter block contains the configuration parameters required for fine-grained tuning of resource allocation/management and network configuration of the monitor. There are ten parameters and four sub-blocks a mos block can hold:

• (Mandatory for DPDK) nb_mem_channels parameter specifies the number of memory channels per socket. Please refer to DPDK user guide, when you need to tune this parameter for performance optimization.
• (Mandatory) netdev parameter block specifies the network interfaces (or device ports) used for mOS applications. Each line should contain the mapping of a network interface and CPU core mask. In the following example, it uses two DPDK-assisted network interfaces (dpdk0 and dpdk1), and CPU cores 0~3 receive/transmit traffic from/to the first 4 (0~3) hardware queues of the NIC. We program our DPDK driver to use symmetric RSS algorithm to distribute traffic across the cores. This ensures that the same CPU gets to examine packets of both flows of the connection.

netdev {
        dpdk0 0x000F
        dpdk1 0x000F
}

• (Optional but recommended) stat_print parameter specifies the network interfaces which the application is interested in displaying runtime monitor statistics. In the example below, although there are two network interfaces, it will only print out the network statistics for dpdk1.

netdev {
        dpdk0 0x000F
        dpdk1 0x000F
}
...
stat_print = dpdk1

• (Mandatory) mos_log parameter specifies the path to a directory where the mOS system writes extra logging data.
• (Optional) arp_table parameter block specifies the static arp table entries. Although mOS retrieves arp table entries from the kernel during initialization stage, the user can add additional static arp table entries if he/she wants. Each line consists of destination IP address and destination MAC address. The following example shows how to set up user-space static arp table entries (If you don’t want static arp table setup, you can simply write an empty arp_table block.). Please note that the mOS networking core does not learn ARP entries dynamically.

arp_table {
        10.0.0.3/32 00:1b:21:81:c1:8c
        10.0.0.4/32 a0:36:9f:2c:c2:84
}

• (Optional) route_table parameter block specifies the static route table entries. Although mOS will retrieve the route table from kernel in the initialization stage, the user can add additional route table entries here. Each line consists of destination IP address and destination interface port. The following example shows how to set up the static route table (If you want no static route table setup, you can simply write an empty route_table block.).

route_table {
             10.0.0.0/24 dpdk0
}

• (Optional) nic_forward_table parameter block specifies the static Ethernet traffic forwarding rules when the mOS middlebox application is set in inline mode (Running Monitor Applications in Inline Mode). If enabled, each line accepts a pair of DPDK-registered NIC interfaces via which the traffic can later be forwarded in either direction. Each interface should have one-to-one mapping with another. In the example below, traffic is switched from dpdk0 to dpdk1 and vice versa. The middlebox is configured as a ‘bump-in-the-wire’.

nic_forward_table {
             dpdk0 dpdk1
}

• (Optional) forward parameter specifies the global policy for forwarding packets in the monitoring applications. You can disable forwarding the packets (e.g., in passive monitoring setup) by putting forward = 0. [default value: 1]

Attention

Please set forward parameter to zero for mOS TCP (mTCP) applications (e.g., epserver or epwget), since they should not forward the incoming packets to the network. Even if you set forward = 1 for mOS TCP (mTCP) applications, it will ignore the parameter and print a warning message.

• (Optional) max_concurrency parameter specifies the maximum number of concurrent flows a middlebox can examine per CPU core. This parameter is used for preallocating the memory for flows. [default value: 100000]
• (Optional) no_ring_buffers parameter determines whether the TCP ring buffer should be disabled. For mOS TCP (mTCP) applications, mOS always require the socket buffers for the application. For middlebox applications, you can disable the receive-side socket buffer by putting no_ring_buffers = 1. [default value: 0 (ring buffer is enabled by default)]
• (Optional) rmem_size parameter specifies the size of receive buffer per socket in bytes [default value: 8192 B].
• (Optional) wmem_size parameter specifies the size of send buffer per socket in bytes [default value: 8192 B].
• (Optional) tcp_tw_interval parameter specifies the TCP timewait interval value in seconds. TCP timewait interval is the allowed time for a connection to be in the TIME_WAIT state. For mOS TCP (mTCP) applications, you can set it as an arbitrary value larger than 0 to guarantee graceful shutdown. For monitoring applications, it is your choice to set this parameter. You may set it to 0 if you’re not interested in monitoring the flow after TIME_WAIT state. [default value: 0s]
• (Optional) tcp_timeout parameter specifies the TCP timeout value in seconds. This parameter determines the maximum allowed time for any flows to exist without any packet reception. In other words, mOS middlebox stops monitoring the connection after tcp_timeout period. You can set tcp_timeout = -1 to disable the timeout checking. For mOS TCP (mTCP) applications, tcp_timeout serves as a timeout period for connections that are in active open states (SYN_SENT states). [default value: 30s]