Network Address Translation (NAT) ================================== Introduction ------------ The ``nat`` program is a sample Network Address Translation (NAT) program which maps a single IP address to multiple network addresses. This application can be used for sharing one global IP address of a NAT gateway for an entire private network. The following code block shows an example run of a ``nat`` program. The user provides a global IP address to be shared using ``-i`` parameter. The ``-c`` parameter directs the mOS core to run the application in ``num_cores`` number of cores. .. code-block:: c nat -i 200.200.200.200 -c .. caution Please remember to add the ARP entry for the translated address in the middlebox as well. Code Walkthrough ----------------- The following sections provide an explanation of the main components of the ``nat`` code. All mOS net library functions used in the sample code are prefixed with ``mtcp_`` and are explained in detail in the `Programmer's Guide - mOS Programming API`_. Note that we omit the error handling logic from the code snippets for brevity. (1) The main() Function ~~~~~~~~~~~~~~~~~~~~~~~~~ The ``main()`` function performs the initialization and calls the execution threads for each CPU core. The first task is to initialize mOS thread based on the mOS configuration file. ``mos_conf_file`` is the file path to the ``mos.conf`` file which will be provided to ``mtcp_init()``. .. code-block:: c /* parse mos configuration file */ ret = mtcp_init(mos_conf_file); In case additional changes are required in mOS configuration (e.g., updating the number of cores), we can use ``mtcp_getconf()`` to first retrieve the config. It then calls ``mtcp_setconf()`` function to update the configuration with new settings. In the case of ``nat``, we update the ``num_cores`` variable that is passed by the user as a command-line argument. .. code-block:: c /* set the core limit */ mtcp_getconf(&mcfg); mcfg.num_cores = g_core_limit; mtcp_setconf(&mcfg); Next, it allocates the data structure for storing port numbers. ``g_free_addrs`` is a queue that consists of free addresses, which are not yet allocated. Since ``g_free_addrs`` is a global queue shared among multiple CPU core threads, it should be protected by the global mutex lock named ``g_addrlock``. The ``nat`` program does not use the reserved port number space (from 0 to 1024). .. code-block:: c /* Initialize global data structure */ pthread_mutex_init(&g_addrlock, NULL); TAILQ_INIT(&g_free_addrs); for (i = 1025; i < 65535; i++) { struct port *p = malloc(sizeof(struct port)); p->port = htons(i); TAILQ_INSERT_TAIL(&g_free_addrs, p, link); Afterwards, the main function creates mtcp context for each core and launches the per-core thread starting from the function ``init_monitor()``. ``init_monitor()`` function will be discussed in the following subsection. .. code-block:: c for (i = 0; i < g_max_cores; i++) { /* Run mOS for each CPU core */ if (!(g_mctx[i] = mtcp_create_context(i))) { ... } /* init monitor */ init_monitor(mctx_list[i]); } (2) Per-thread Initialization Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``init_monitor()`` function is the core functional part of the mOS thread initialization. In order to monitor the ongoing TCP flows, this application creates a ``MOS_SOCK_MONITOR_STREAM`` socket as follows. .. code-block:: c /* create socket */ lsock = mtcp_socket(ctx->mctx, AF_INET, MOS_SOCK_MONITOR_STREAM, 0); Using the ``MOS_SOCK_MONITOR_STREAM`` socket named ``lsock``, ``nat`` registers the callback functions for those TCP events that it is interested in. First, ``translate_addr()`` function is called whenever a new packet arrives. Second, ``release_port()`` function is called whenever the connection finishes. The ``translate_addr()`` and ``release_port()`` functions will be described in further detail in the later subsections. .. code-block:: c mtcp_register_callback(mctx, lsock, MOS_ON_PKT_IN, MOS_HK_SND, translate_addr); mtcp_register_callback(mctx, lsock, MOS_ON_CONN_END, MOS_HK_SND, release_port); (3) The ``translate_addr()`` Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``translate_addr()`` function is used for translating the network addresses in a packet-by-packet manner. First, it tries to call ``mtcp_get_uctx()`` to check whether the address of the flow has already been translated . If it is a first packet from the flow (meaning that the private user context ``uctx`` is NULL), it will try to assign a new port number. .. code-block:: c if (!(w = mtcp_get_uctx(mctx, sock))) assign_port(mctx, sock); When it assigns a port using the ``assign_port()`` function, it will first retrieve the network address of both sides, find the appropriate port number, and pull the port number from the free port number list (to use it). In order to determine the appropriate port number translation, it uses ``GetRSSCPUCore()`` function provided by mOS, so that the translated flow maps to the same CPU core as before the translation. .. code-block:: c mtcp_getpeername(mctx, sock, (struct sockaddr *)&addr, &len, MOS_SIDE_CLI); /* hold the mutex lock g_addrlock before accessing the g_free_addrs */ pthread_mutex_lock(&g_addrlock); /* find for an appropriate port number to be mapped (to meet core affinity) */ TAILQ_FOREACH(w, &g_free_addrs, link) if (GetRSSCPUCore(g_NATIP, addr[MOS_SIDE_SVR].sin_addr.s_addr, w->port, addr[MOS_SIDE_SVR].sin_port, g_core_limit) == mctx->cpu) break; TAILQ_REMOVE(&g_free_addrs, w, link); /* release the lock */ pthread_mutex_unlock(&g_addrlock); (4) The ``release_port()`` Function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When the connection finishes, the ``release_port()`` callback function will be triggered. The role of ``release_port()`` function is simply to insert the port number back in to the free port number list (``g_free_addrs``), and set the port number allocation metadata to ``NULL`` (by passing ``NULL`` to ``mtcp_set_uctx()``) to release the port. .. code-block:: c if (!(w = mtcp_get_uctx(mctx, sock))) return; /* assign a port number */ pthread_mutex_lock(&g_addrlock); TAILQ_INSERT_TAIL(&g_free_addrs, w, link); mtcp_set_uctx(mctx, sock, NULL); pthread_mutex_unlock(&g_addrlock); .. important:: Due to the scalability limitations of our nat application, please restrict the maximum concurrent connections limit to at most 2000 flows at any given time. .. _`Programmer's Guide - mOS Programming API`: ../programmer/04_mos_api.html