mOS Event System ================ The core part of mOS is its event module that facilitates event-driven middlebox processing for monitoring applications. The mOS API encourages modular middlebox programming by converting a complex application into a set of independent pairs. Our experience in programming applications with mOS API led us to the conclusion that existing middlebox applications' logic can easily be transformed into set of mOS defined events and their corresponding handlers. mOS provides two classes of events: *(i)* built-in and *(ii)* user-defined events. Built-in Events ~~~~~~~~~~~~~~~ Currently, mOS provides 8 built-in events that the developer can use. =========================== ================================ Built-in event Description ``MOS_ON_PKT_IN`` In-flow TCP packet arrival ``MOS_ON_CONN_START`` New connection initiation ``MOS_ON_REXMIT`` TCP retransmission ``MOS_ON_TCP_STATE_CHANGE`` TCP state change ``MOS_ON_CONN_END`` Connection termination ``MOS_ON_CONN_NEW_DATA`` New flow payload ``MOS_ON_ORPHAN`` non-TCP packet ``MOS_ON_ERROR`` Error report (e.g., receive buffer full) =========================== ================================ .. note:: 1. mOS raises ``MOS_ON_CONN_END`` event when TCP stream on each side is destroyed. 2. mOS raises ``MOS_ON_CONN_START`` event when it observes a TCP SYN packet. A mOS application can register one callback function per built-in event on each socket. With a registered callback, a user can retrieve attributes of an ongoing connection from the mOS core. These attributes can be packet information (that triggers the callback function), the state information of the corresponding TCP flow (TCP state, received buffer contents/offset) *etc.* We suggest you to refer to the `mOS Programming API`_ for details. User-defined Events (UDEs) ~~~~~~~~~~~~~~~~~~~~~~~~~~ Some middlebox developers may find built-in events insufficient for implementing the applications' logic. For example, an event that gets triggered after detecting an HTTP request does not exist in mOS by default. Such an event would require mOS to parse L7 (HTTP) headers of ingress packets. Although a developer can use built-in events (*e.g.*, ``MOS_ON_PKT_IN``, ``MOS_ON_CONN_NEW_DATA``) to develop her own logic that covers all the checks for such complex events, there is always a tangible risk of the event code (for detecting *conditions* of middlebox processing) getting strongly coupled within the application logic. This design can lead to overly complex codebase that quickly becomes difficult to maintain over time. User-Defined Events (UDEs) allow the developers to create their own events (that don't already exist in the built-in events list). A UDE is defined using a base event and a boolean filter function that specifies the event condition. When the base event is raised, the mOS stack evaluates the filter function and raises the UDE only if the filter returns true. UDEs bring three advantages to the event-driven middlebox development. First, new types of events can be created in a flexible manner because the filter function can evaluate arbitrary conditions of interest. A good filter function, however, should run fast without producing unnecessary side effects. Second, UDEs provide easy extensibility. One can create new events by extending any existing ones, including another UDE. For example, a developer can define a UDE that detects a Google search query by extending a generic HTTP request UDE. Third, it encourages code reuse. One can share a well-designed set of event definitions as a UDE library, and third party developers can implement their own event handlers. For example, an open-source NIDS can declare all corner cases in flow management as a UDE library while third party can provide custom actions to address each case. Event Hook and Flow of Events ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. figure:: images/pkt_processing.png :scale: 20 % :align: center Figure 3.1. Flow of Events in mOS stack Figure 3.1 described the sequence of events in an inline mOS monitor and shows how an ingress packet is processed in the mOS stack. On entry, the flow corresponding to the packet is first identified. In case the packet is TCP SYN, a new flow entry is created. The mOS core then updates the sender side's TCP stack. Once the stack has been updated, it raises all flow events that are registered by the monitoring application pertaining to the sender side. It then checks whether the TCP packet is retransmitted. Finally it repeats the same process with receiver side stack before the packet is forwarded. .. attention:: In the mOS stack, client and server can both be sender and receiver. For example, a SYN packet transmission from the client (to the server) would make client a sender and the server a receiver. On the flip side, a SYN-ACK packet would have the server acting as a sender while a client as the receiver. We have placed two event hooks from where mOS events can be triggered. A developer can use ``MOS_HK_SND`` to capture events from sender's side while ``MOS_HK_RCV`` can be used to receive events from the receiver side. .. important:: Both event hooks are only triggered **after** the respective states have been updated. Table 3.2 shows which events are available at given hook points in mOS. Please note that ``MOS_NULL`` hook point is used for those networking events that are raised without any flow context. ``MOS_ON_PKT_IN`` may still use ``MOS_NULL`` hook for those packets that don't have a corresponding flow context (*e.g.*, a TCP packet that arrives without a TCP handshake). ``MOS_ON_CONN_NEW_DATA`` is a special event that gets triggered when the mOS core detects that a flow has received reassembled payload in the receive buffer. This event can come out of context since the mOS core uses the DPDK I/O driver to read batch of packets from the NIC queue. The event is only raised once all the packets in the batch have been processed. This helps the mOS core to amortize the cost of calling ``MOS_ON_CONN_NEW_DATA`` event handler by triggering the event only once when multiple data packets arrive for the same flow. =========================== ============================================ Event Available hooks ``MOS_ON_PKT_IN`` ``MOS_NULL``, ``MOS_HK_SND``, ``MOS_HK_RCV`` ``MOS_ON_CONN_START`` ``MOS_HK_SND``, ``MOS_HK_RCV`` ``MOS_ON_REXMIT`` ``MOS_HK_SND``, ``MOS_HK_RCV`` ``MOS_ON_TCP_STATE_CHANGE`` ``MOS_HK_SND``, ``MOS_HK_RCV`` ``MOS_ON_CONN_END`` ``MOS_HK_SND``, ``MOS_HK_RCV`` ``MOS_ON_CONN_NEW_DATA`` ``MOS_NULL`` ``MOS_ON_ORPHAN`` ``MOS_NULL`` ``MOS_ON_ERROR`` ``MOS_NULL`` =========================== ============================================ Table 3.2: List of events and their available hook points. .. note:: Some events can be registered both on ``MOS_HK_SND`` and ``MOS_HK_RCV`` hooks. Examples include ``MOS_ON_PKT_IN``, ``MOS_ON_CONN_START`` and ``MOS_ON_TCP_STATE_CHANGE``. In order to distinguish these events from each other, all mOS event handlers pass ``side`` (client/server) variable as a function argument. Example Initialization Sequence ------------------------------- A code snippet below shows all the concepts that we have introduced in this section. .. code-block:: c enum {MOS_SIDE_CLI = 0, MOS_SIDE_SVR}; /* * mOSAppInit(): * [In]: mctx_t m - Per-thread mOS context */ static void mOSAppInit(mctx_t m) { monitor_filter_t ft = {0}; int s; event_t hev; // create a passive monitoring socket & set up its traffic scope s = mtcp_socket(m, AF_INET, MOS_SOCK_MONITOR_STREAM, 0); if (s == -1) { perror("mtcp_socket() failed: "); exit(EXIT_FAILURE); } ft.stream_syn_filter = "dst net 216.58 and dst port 80"; if (mtcp_bind_monitor_filter(m, s, &ft) == -1) { perror("mtcp_bind_monitor_filter() failed: "); exit(EXIT_FAILURE); } // set up a built-in event handler for MOS_ON_REXMIT if (mtcp_register_callback(m, s, MOS_ON_REXMIT, MOS_HK_RCV, OnRexmitPkt) < 0) goto register_err; // set up a built-in event handler for MOS_ON_PKT_IN if (mtcp_register_callback(m, s, MOS_ON_PKT_IN, MOS_HK_RCV, OnFlowPkt) < 0) goto register_err; // set up a built-in event handler for MOS_ON_TCP_STATE_CHANGE if (mtcp_register_callback(m, s, MOS_ON_TCP_STATE_CHANGE, MOS_HK_RCV, OnTcpStateChange) < 0) goto register_err; // set up a built-in event handler for MOS_ON_CONN_END if (mtcp_register_callback(m, s, MOS_ON_CONN_END, MOS_HK_RCV, OnFlowEnd) < 0) goto register_err; // define a user-defined event that detects an HTTP request hev = mtcp_define_event(MOS_ON_CONN_NEW_DATA, IsHTTPRequest); if (hev == 0) { perror("mtcp_define_event() failed: "); exit(EXIT_FAILURE); } // set up an event handler for hev if (mtcp_register_callback(m, s, hev, MOS_HK_RCV, OnHTTPRequest) < 0) goto register_err; register_err: perror("mtcp_register_callback() failed: "); exit(EXIT_FAILURE); } /* * OnFlowPkt(): * [In]: mctx_t m - Per-thread mOS context * [In]: int sock - active monitoring socket (represents individual connection) * [In]: int side - Server/Client? * [In]: event_t event - the event that was triggered */ static void // callback for MOS_ON_PKT_IN OnFlowPkt(mctx_t m, int sock, int side, event_t event, filter_arg_t *arg) { if (side == MOS_SIDE_CLI) g_pktcnt[sock]++; } static void // callback for MOS_ON_REXMIT OnRexmitPkt(mctx_t m, int sock, int side, event_t event, filter_arg_t *arg) { g_rexmit_cnt[sock]++; } static void // callback for MOS_ON_TCP_STATE_CHANGE OnTCPStateChange(mctx_t m, int sock, int side, event_t event, filter_arg_t *arg) { if (side == MOS_SIDE_CLI) { int state; socklent_t len = sizeof(state); if (mtcp_getsockopt(m, sock, SOL_MONSOCKET, MOS_TCP_STATE_CLI, &state, &len) < 0) { perror("mtcp_getsockopt() failed: "); return; } if (state == TCP_FIN_WAIT_1) g_cli_term[sock]++; } } static void // callback for MOS_ON_TCP_CONN_END OnFlowEnd(mctx_t m, int sock, int side, event_t event, filter_arg_t *arg) { if (sock != MOS_SIDE_CLI) return; TRACE_LOG("TCP flow (sock=%d) had %d packets, rexmit: %d\n", sock, g_pktcnt[sock], g_rexmit_cnt[sock]); g_pktcnt[sock] = 0; g_rexmit_cnt[sock] = 0; g_total_flows++; } .. _`mOS Programming API`: ./04_mos_api.html