Running mTCP applications with mOS ================================== This section shows one can use mOS library to run mTCP_ networking applications. We describe how one can set up sample mOS web server and client, and configure them to talk to each other. .. note:: We have ported a couple of mTCP programs in mOS. These applications are functionally equivalent to those that are currently available in the `mTCP release`_. .. The user is advised to refer to the `Configuration Parameter Tuning`_ page to know .. further about the more fine-grained configuration details that are not mentioned .. in this section. Environment ----------------------- This section assumes an environment configured as described below. .. figure:: images/endtcp_config.* :align: center :scale: 50 % * Data Network: Any Ethernet network that can deliver traffic between two TCP endpoints. * Host 1: This host runs a :doc:`../samples/epwget` instance, that runs on mOS stack. It has one NIC named ``dpdk0``, which is connected to the Data Network. * Host 2: This host runs a :doc:`../samples/epserver` instance, that runs on mOS stack. It has one NIC named ``dpdk0``, which is connected to the Data Network. Configuration Steps ----------------------- Compile and build mOS library and applications ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Follow :doc:`../walkthrough/03_setup` to set up the mOS net library and the environment. * In Step 3 of Section 3.2.3, please note that you need to assign a valid IP address to network interface to allow ``dpdk0`` in each host to communicate with each other over the Data Network. Refer to :ref:`sample_app`, and make sure that you successfully compile the mOS applications on both endpoints. * In Host 1, you should compile `epwget`. * In Host 2, you should compile `epserver`. Setup mOS application configurations ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Before running the ``epwget`` and ``epserver`` applications, you need to configure the configuration files in each host. * In Host 1, please configure ``epwget.conf`` as below. .. code-block:: c url = (IP address of Host 2)/(file name) dest_port = 80 total_flows = (number of total flows to generate) total_concurrency = (number of concurrent flows to run) core_limit = (number of CPU cores to run) * In Host 1, please configure ``mos.conf`` as below. We assume that Host 1 uses CPU core 0 to 7 for running mOS ``epwget``. .. code-block:: c ############### MOS configuration file ############### ####################### # APPLICATION OPTIONS # ####################### # application to run (epserver) application { type = end run = epwget core_mask = 0x00FF # 0-7 } ####################### ## mos configuration ## ####################### mos { ####################### ### TCP/IP OPTIONS #### ####################### # maximum concurrency per core # (mos-specific parameter for preallocation) max_concurrency = 100000 # maximum number of socket buffers per core max_num_buffers = 100000 # disable the ring buffer no_ring_buffers = 0 # receive buffer size of sockets rmem_size = 8192 # send buffer size of sockets wmem_size = 8192 # tcp timewait seconds tcp_tw_interval = 0 # tcp timeout seconds # (tcp_timeout = -1 can disable the timeout check) tcp_timeout = 30 ####################### ##### I/O OPTIONS ##### ####################### # number of memory channels per socket ( dpdk - only ) nb_mem_channels = 4 # interfaces the mOS have (with core mapping) netdev { dpdk0 0x00FF # 0-7 } ####################### ### LOGGING OPTIONS ### ####################### # A directory contains log files mos_log = logs/ # NICs to print network statistics per second # if enabled, mTCP will print xx Gbps and xx pps for RX and TX stat_print = dpdk0 } * In Host 2, please configure ``epserver.conf`` as below. .. code-block:: c www_main = (a folder path which contains the file to serve) core_limit = (number of CPU core to run) * In Host 2, please configure ``mos.conf`` in the same manner that was mentioned for Host 1 with the exception of the following lines: .. code-block:: c # application to run (epserver) application { type = end run = epserver core_mask = 0x00FF # 0-7 } We also assume that Host 2 uses CPU core 0 to 7 for running the mOS ``epserver``. .. important:: 1. mOS ignores the ``forward`` option for mTCP applications. 2. Please avoid using PCAP I/O module for mTCP applications. 3. For optimal performance, please adjust the configuration parameters such as ``total_concurrency`` (in ``epwget.conf``), ``max_concurrency``, ``max_num_buffers``, ``rmem_size``, or ``wmem_size`` (in ``mos.conf``) by referring to :doc:`../walkthrough/05_configuration`. Run epserver and epwget ------------------------- In Host 2, run ``epserver`` first with sudo privileges. .. code-block:: console $ sudo ./epserver In Host 1, when you run ``epwget`` with sudo privileges, you will see that ``epwget`` of Host 1 downloads the files from ``epserver`` running on Host 2. .. code-block:: console $ sudo ./epwget Troubleshooting ----------------------- If your mOS program crashes, search through the printed log messages to check if you missed any of the following requirements: * You should load only the dpdk-registered interfaces (after loading the ``igb_uio`` driver) and configure hugepages as decribed in :doc:`../walkthrough/03_setup`. * You should create a log folder specified in ``mos_log`` parameter in ``mos.conf`` (``logs/`` in our example) within the same directory that contains the mOS application binary. * Please check if your system is running out of memory by looking at the memory footprint (OOM errors). If you have memory deficiency issues, please reduce the number of socket buffers in ``mos.conf``. If the HTTP connections between Host 1 and 2 are lost, check the following items: * Make sure that Host 1 and Host 2 have full network connectivity over the interface that runs on default Linux Ethernet driver. Please note that mTCP networking stack uses the ARP and route tables from the kernel. * In other words, you can check the connectivity by going back to kernel network module (see the option 5 in :ref:`kernel_module`) and running default web server and client (e.g., nginx and wget) programs. .. warning:: Please note that the mOS stack currently can only run on a CPU processor with 16 CPU cores or less. This restriction will be removed in future versions of the stack. For now, we suggest that the user disables all CPU core IDs > 16 using the ``sysfs`` interface. .. code-block:: console $ cd /sys/devices/system/cpu $ for i in {16..$MAX_CPU} do cd $i echo 0 > online cd .. done .. _Configuration Parameter Tuning: 05_configuration.html .. _Sample Applications User Guide: ../samples/index.html .. _`3.2.3`: ./03_build_inline.html#decompress-mos-package-on-m .. _`mTCP release`: https://github.com/eunyoung14/mtcp .. _`mTCP`: http://shader.kaist.edu/mtcp/