Evaluating contacts in opportunistic networks over more realistic simulation models

Opportunistic Networks (ONs) are mobile networks that support intermittent links and long delays. ON nodes exchange data in brief moments called contacts, when another node is within radio range. Contacts are ephemeral and unpredictable, thus they must be implemented as efficiently as possible. However, most previous work rely on simplistic assumptions such as unlimited bandwidth and contentionfree transmissions. This paper presents a more realistic evaluation of ON contacts. Simulations show that, on opposition to the consensus in the literature, routing protocols that forward more copies and those that determine a subset of nodes to receive the Bundles using a certain criteria outperform flooding-based protocols, because the latter generates too much medium contention. Finally, buffer management and forwarding prioritization may influence the performance of the network by up to 30%.


Introduction
Opportunistic networks (ONs) are used to establish communication without requiring any network infrastructure (access points, antennas, etc) in situations where and instantaneous end-to-end path is not available (Pelusi et al., 2006).Those networks are characterized by the forwarding of messages among mobile nodes, from source to destination, using wireless links.
Those networks are usually sparse, creating disconnected network topologies.This is due to the limited range of wireless radios, node mobility, and small number of nodes as well as the tendency of nodes to move according to certain patterns.In ONs, traditional message passing techniques do not perform well because they require an instantaneous end-to-end path from source to destination (Fall, 2003).Opportunistic networks require a communication paradigm that is tolerant to disruptions and long delays, such as the DTN (Delay and Disruption Tolerant Networks) paradigm.In this paradigm, self-contained messages called Bundles are stored in intermediate nodes until they can be forwarded again.
In ONs, nodes exchange Bundles whenever they are in radio range, in an operation called contact.Contacts have different durations, and typically are unpredictable.As such, it is in the best interest of the network to maximize the amount of data transmitted on each contact, by exchanging Bundles or connectivity information with the most suitable nodes in routing (Manfredi et al., 2011), by prioritizing the Bundles to be forwarded (Lindgren and Phanse, 2006), or by ensuring that the Bundles stored in the buffers have a high likelihood to be delivered (Lindgren and Phanse, 2006).
Most ON studies dedicated to contact evaluation, however, do not employ realistic MAC and PHY models, assuming that the communication is reliable and error-free.However, most wireless standards employ channel access schemes that are prone to collisions when multiple mobile nodes transmit at the same time.Further, the wireless medium is ripe with sources of error Rappaport (2002).Moreover, it has been reported that evaluation studies based on simplistic models tend to overestimate the performance of the network by up to 300% (Ristanovic et al., 2012).

Evaluating contacts in opportunistic networks over more realistic simulation models
This paper makes the case for more realistic PHY and MAC models in ON simulations.We evaluate the performance of several queuing and forwarding strategies over different routing protocols.The results show that the behavior of the network changes significantly from what has been reported in previous studies, due to effects such as collisions, channel contention and transmission errors.
The remainder of the paper is organized as follows.First, we overview the related work.The evaluated contact models are described next, followed by a description of the details the simulation setup and the discussion of the results.Finally, we present the conclusions and future work.

Related work
Although contacts are an important aspect of ONs, most studies tend to overlook them.The ONE simulator (Keränen et al., 2009) is the most popular ON simulator, however, it does not model important characteristics of the PHY and MAC layers such as interference, contention, packet collisions and the cost of exchanging control packets among nodes.The NS-2 (NS-2, 2013) simulator, on the other hand, has more realistic radio and MAC models.It implements popular MAC protocols, and its PHY layer considers collisions, interference, power losses due to signal propagation and fading, as well as packet captures (Whitehouse et al., 2005).However, the simulator does not incorporate any DTN routing protocol or contact mechanism, requiring the users to implement the DTN stack from scratch.
A recent study (Mota et al., 2014) surveyed fifteen articles proposing routing protocols in order to map the use of different models.Out of those protocols, 22 percent employed NS-2, which provides rich MAC and PHY models.The others, on the other hand, employed mostly custom simulators, with only one employing ONE.Due to the complexity of implementing a simulator, we believe that those works employed quite simplistic communication models, ignoring several of the parameters described above.
Manfredi et al. studied how the contact time and distribution influence the choice of routing protocol (Manfredi et al., 2011).They evaluated whether it is more suitable to keep network state to build efficient routes, or to forward multiple copies of the Bundle through different nodes.The study concluded that, in networks with high unpredictability and low connectivity, it is better to employ floodingbased algorithms.For networks with more predictable contacts, informed forwarding decisions based on past node behavior perform best.Finally, for networks with high connectivity and low unpredictability, it is better to exchange control messages and build end-toend routes.Lindgren and Phanse (2006) investigate the contact from the point of view of the link layer.They evaluate different buffer management techniques and forwarding prioritization schemes on the performance of DTN routing protocols.The authors evaluate which Bundles must be discarded when the buffers are full.Further, the forwarding prioritization schemes define which Bundles should be sent first upon a contact, in an attempt to maximize the overall delivery ratio of the network.Ying et al. (2013) presented a buffer management scheme where Bundles are classified according to the validity of the data, so that Bundles are discarded when users have no more interest in it (Ying et al., 2013).Another proposed approach is to classify Bundles into multiple queues, based on their destination and priority (Rashid et al., 2013).However, it is very hard in practice to classify those two parameters in a Bundle.
Besides the queuing policy, the size of the buffers is important for the network performance.A buffer dimensioning approach for nodes moving following the Random Waypoint mobility pattern was presented in Bononi et al. (2012).Mahendran and Murthy devised an analytical model for buffer sizing, based on Markov modulated processes, for Epidemic-like protocols (Mahendran and Murthy, 2013).
Those studies, however, did not employ realistic MAC and PHY models.For example, they ignores the effects of collisions and contention, as well as the message dialogue of the MAC layer.
Building realistic simulations is hard.Due to the omnipresence of ONE, most existing traces (i.e.Crawdad, 2013) do not log location and speed, which are necessary to simulate PHY effects of signal attenuation and fading.Further, it has been shown that the mobility models must be carefully parameterized in order to provide results similar to a trace-based evaluation (Thakur et al., 2011).
Further, Ristanovic et al. showed the gap between trace-based simulation studies and real experiments (Ristanovic et al., 2012).They concluded that the delivery ratio and the delay are underestimated by 2-3 times in simulations due to the use of simplistic models.They blame the assumptions of unlimited bandwidth, error-free channels, and instantaneous contacts.
This paper contributes to the case of more realistic simulations by showing the effects of a more accurate MAC and PHY layer on contact performance.Our simulation takes into account more communication parameters than previous works, as follows: medium contention and the use of back-offs in CSMA/CA; the packet capture effect (Whitehouse et al., 2005); a complete implementation of a contact protocol, including exchanging the list of Bundles and requesting the bundles to be forwarded; contact duration based on the signal quality at the receiver; routing information exchange among nodes employing messages instead of perfect data exchange; probe-based detection of neighbors.We reanalyze the results of Lindgren and Phanse ( 2006), showing significant differences in the results when those factors are taken into account.

Network model
This paper evaluates the forward and discard policies of Lindgren and Phanse (2006), which are described in the following sections.We also implemented our own contact protocol, described below.

Contact protocol
Our contact implementation is based on periodic polling.The first step is the announcement message, which indicates to other nodes that a given node is ready to exchange Bundles.Nodes within radio range will receive this message, identifying that this node is reachable.The announcement contains the list of stored Bundles that may be forwarded.Bundles that cannot be forwarded (for example Bundles in the Wait phase of Spray & Wait) are not announced in order to reduce the overhead1 .
If a node wishes to receive Bundles in custody of another node, it replies to the announcement, listing the requested Bundles.
The sender, then, will forward the Bundles based on a forwarding strategy.Finally, the request is followed by the transmission of the requested Bundles.
The performance of our contact depends on the period among announce messages, which must be a tradeoff of control traffic overhead (more announce messages also mean more congestion) and the delay to discover a new node.

Forwarding strategies
The forwarding strategies dictate when a Bundle will be forwarded, as well as its priority.This is necessary since the contact may end before all the Bundles have been transmitted, for example due to node mobility or interference.Those strategies can be employed in informed forwarding (Chaintreau et al., 2007) (e.g.ProPHET) routing protocols, which compute the contact probabilities based on previous contacts.Suppose that node A wants to send a message to D, and contacts node B. Further, the probability of node X contacting node Y is given by P X,Y . 

Discard policies
The discard policies dictate the bundle to be removed from a full buffer, in order to make room for a new Bundle. 

Routing protocols
Since we focus our work on ONs, we evaluated only stochastic DTN routing protocols.Stochastic protocols can be divided into three categories (Chaintreau et al., 2007): flooding, where the messages are disseminated to all nodes upon a contact; wait-and-forward, where a limited number of messages are disseminated; and informed forwarding, where the choice of nodes is based on previously acquired knowledge.Our objective in this evaluation is to present the general trends that occur in the classes of routing protocols, not the results of the state of the art on each class.Although more refined routing protocols already exist in the literature, the performance of the contact will follow similar trends among the categories.Thus, by being generic in our analysis instead of focusing on the specific techniques employed by each protocol, the analysis applies to a wider range of protocols and networks.We employed Epidemic as the flooding-based algorithm; Spray & Wait represents the wait-and-forward class; and PRoPHET was selected among the informed forwarding protocols.They are classic DTN protocols, which are briefly described below.
Flooding: Epidemic -is a stochastic routing algorithm for DTNs where the source node sends the message to all its neighbors, which then send the message to all their neighbors (Vahdat and Becker, 2000).This cycle is repeated up to a maximum number of hops, determined by the Time-to-live (TTL) field of the message.Using node mobility, the message can be delivered to regions of the network that are not reachable through and instantaneous end-to-end path.Epidemic routing is very efficient in terms of its delivery ratio, however the number of messages sent increases proportionally to the number of nodes (Vahdat and Becker, 2000).
Wait-and-forward: Spray & Wait -Spray and Wait uses two forwarding strategies (Spyropoulos et al., 2005).In the first strategy, called Spray, L message copies are disseminated.The source node forwards the message to all its neighbors, which store the message in their buffer, and then forward the message.Each neighbor is allocated a fraction of L. This process is repeated several times, and at each time the value of L is reduced.When a node receives the message having L set to zero, the process stops.The destination may not be reached using the Spray strategy.Hence, nodes may use the Wait strategy: the message is forwarded directly to the destination if one of the nodes having the message in its buffer contacts the destination.
Informed forwarding: PRoPHET -Probabilistic Routing Protocol using History of Encounters and Transitivity forwards messages according to an expected delivery probability, based on connectivity analysis (Lindgren et al., 2003).PRoPHET assumes that node movements are not entirely random, thus the protocol forwards messages to nodes that make more frequent contacts with others, which are more likely to contact the destination.

Evaluation
This section describes the simulations performed in order to evaluate the contact implementations.

Simulation Setup
The simulation employs the NS-2 simulator, since it provides realistic MAC and PHY models.We chose not to employ DTN simulators because they abstract the physical and link layers, assuming that packets do not suffer interference, there is no medium contention.Further, it is common in DTN simulators to assume an infinite bandwidth.Thus, we chose to employ a wireless network simulator such as NS-2.NS-2 is well established in the wireless networking community.Its wireless module has been developed around the year 2000, and has received significant scrutiny and contributions over the years.We did not employ NS-3 because at the time that our work begun, NS-3 had less features than NS-2 and was still regarded as an experimental simula-tor.Other simulators such as OMNet++ and GLOMOSIM could be used, however, they have the same capabilities of NS-2.
NS-2 has been adapted for ONs by implementing the proposed contact mechanism as well as existing DTN routing protocols.The evaluated scenario mimics a vehicular network, in which Bundles describing traffic conditions and road security are disseminated among the nodes (Franck and Gil-Castineira, 2007).Nodes start the simulation with 1000J of energy.Only the energy consumption of the wireless card is measured.Nodes employ IEEE 802.11b (Cisco, 2013) cards, described in Table 1.A new Bundle is sent every 5s.All Bundles have a fixed size of 10 KB.The Bundle deadline is set to 600s.
The routing protocols were configured as follows.We employed the default parameters for PRoPHET.For Spray and Wait, the maximum number of copies was set to 16, and we employed its binary mode.The interval among announcements (the beacons that identify new contacts) was set to 10s.All parameters of Spray and Wait were set empirically, in an attempt to maximize its delivery rate and reduce its energy consumption.The results are presented with a confidence interval of 95%.Since there is no consensus in the networking literature about what constitutes an acceptable or reliable confidence interval, we defined this value empirically.
Nodes move according to the Random Way Point (RWP) model, where each node chooses with uniform probability a destination in the simulation area, as well as its speed.Upon arriving at the destination, the node pauses for a uniformly distributed time, and then selects another destination.We employed the RWP implementation of Le Boudec and Vojnovic, (2006), which ensures the existence of a stationary state.We deploy 150 nodes distributed on a 2400x2400 area, moving in the area respecting the speed limits of streets and avenues.Both source and destination points are chosen based on a uniform distribution.

Flooding
In the RO literature, Flooding-based protocols usually achieve the highest delivery rate and the lowest delivery delay.This occurs due to the simultaneous forwarding of Bundles to all neighbors, which also incurs in elevated energy consumption.Our simulations showed different trends, due to collisions and medium contention.Thus, it is important to elaborate more why collisions would occur on ONs, since those networks are supposed to be sparse.Collisions occur due to the movement restrictions on urban scenarios (e.g.traffic lights, lack of alternative paths), which generate predictable movement and stop patterns.As an example, several contacts will tend to occur in roundabouts and road crossings (Tournoux et al., 2009), thus generating medium contention.
Figure 1 presents the delivery rate, which decreases with the amount of Bundles sent per second.This occurs due to collisions, which generate packet losses and delays.Figure 2 shows that the amount of collisions increases significantly with increased rates of Bundle creation.Next we analyzed the delay and the number of forwards required to reach the destination, which are presented in Figures 3 and  4, respectively.The delay increases initially, since the medium becomes congested.For larger loads, however, the amount of dropped Bundles increases, and only Bundles that require few forwards to reach the destination are delivered successfully.As a consequence, the average delay of the delivered Bundles is reduced.
When analyzing the discard policies, SHLI outperforms the other policies in delivery delay and delivery rate.Further, MOFO and SHLI deliver messages using one less forward than FIFO, reducing their delivery delay.

Wait and Forward
Protocols based on Wait and Forward (W&F) tend to reduce the communication overhead by avoiding flooding packets.Previous studies identified that those protocols present higher delivery latencies and lower delivery rates than flooding-based protocols, since the Bundles are sent to less nodes.Figures 5 and 6 present the delivery rate and the delay.

Communication range 240m
Energy consumption on transmission 2.25W Energy consumption on reception 1.35W Energy consumption on idle listening 0.075W

Signal output power 10dBm
Table 1.Radio parameters.Unlike the consensus in the literature, W&F presented higher delivery rates and lower delays than Flooding-based protocols, again due to collisions.Thus, a successful delivery in W&F is limited by the mobility pattern of the nodes.The MOFO policy presented the best delivery rate, while FIFO and SHLI presented the best delays.The performance degradations occurred for more than 9 Bundles/second, when the network starts to become congested.Thus, due to this congestion, the amount of dropped Bundles increases, as shown in Figure 7. Collisions also define the trend in energy consumption, as shown in Figure 8.The consumption rises up to the moment when the network becomes congested, and then stabilizes.

Informed forwarding
The Informed Forwarding (IF) class collects information about the nodes and the environment in order to refine the forwarding decisions.This section evaluates the forwarding strategies described previously, since only IF-based protocols calculate the success probability of each forward.The average number of forwards, energy consumption and number of collisions are not presented since they did not vary significantly among the forwarding strategies.
Figures 9 up to 12 present the delivery rate.Except for COIN, the results present similar tendencies.As in the previous classes of routing protocols, the delivery rate deteriorates for heavier workloads.Thus, for this scenario there were no significant differences among the forwarding strategies.However, the random strategy (COIN) reduces the delivery rate by up to 7% for heavier workloads.Thus, the forwarding strategy does not influence the amount of delivered Bundles.
Figures 13 up to 16 present the delay.Again, there are no significant differences among the forwarding strategies, and even COIN presented results similar to the other strategies.SHLI, however, presented slightly better results.

Discussion
The main finding of this article is that, due to a more complete evaluation model, the gains of a more refined buffering policy or the choice of routing algorithm are different from what is the consensus in the literature.This argument is explained in details.
Finding number one: forwarding and discarding policies impact the performance by up to 30%.The obtained results are much lower than that of Lindgren and Phanse (2006).Those differences are due to the use of more detailed models in our simulations.Since we consider the energy consumption of the radio as the overhead metric, while Lindgren and Phanse employ the number of transmitted Bundles, we also identified that idle listening reduces the gains of sending less Bundles.Medium contention was a key aspect in our results, since it tends to limit the amount of forwarded Bundles and the delivery rate due to packet drops.Due to this limitation, we identified differences in the delivery rate of up to 5% for different discard policies, while Lindgren and Phanse obtained improvements of up to 100%.
Finding number two: Flooding-based algorithms are not so good in terms of delay and delivery rates, after all.Previous ON works indicate that Flooding achieves the best delivery rates and delays since it reaches the destination faster than the other classes of routing protocols.Furthermore, Lindgren and Phanse (2006) identified that the performance of Flooding was limited by the buffer size and the forwarding strategy.Our results, on the other hand, show that medium contention penalizes the multiple copies sent by Flooding.As a consequence, Informed Forwarding protocols tend to perform as well as Flooding protocols.Despite the fact that W&F usually present a high delay due to the limited number of forwards, those protocols have results comparable to Flooding for congested networks.
Finding number three: routing is more important than buffering and forwarding strategies on the performance of the network.In our simulations routing was the key parameter for the performance of the network as a whole.While the choice of forwarding policies and buffer discard policies improved the performance of the network by only a few percent, the choice of routing algorithm can double the delivery rate.For example, by using Flooding with a rate of four Bundles per second, the network achieved around 40% of Bundle delivery.Meanwhile, for Wait and Forward techniques, this value surpassed 80%.

Conclusions and future work
The Opportunistic Network (ON) literature tends to model contacts in a simplistic way, assuming communication models that abstract PHY and MAC layer implementations.This work evaluates contacts using more detailed communication models, in order to assess the effect of more realistic models on the    performance of existing protocols.We evaluated contacts considering several forwarding strategies, discard policies and routing protocols.Different from previous results, we verified that flooding-based routing protocols performed worse than informed forwarding protocols, due to collisions caused by medium contention.Furthermore, the energy consumption does not vary significantly among protocols, since the consumption in idle and reception modes is comparable to that of the transmission mode.
The main conclusion of this work is that a more detailed simulation model leads to significantly different results.Effects such as collisions and energy consumption minimize the performance gains of more efficient forwarding algorithms.Thus, it is important to carefully parameterize the simulators in order to obtain more realistic performance figures.
As future work, we plan to improve the simulation scenarios, considering fading, newer communication standards such as IEEE802.11p, and evaluate more routing protocols and mobility models.
(Cerf et al., 2007)es have a lifetime(Cerf et al., 2007), this policy discards the Bundles with the nearest expiration time.This discards Bundles that are the least likely to reach their destination on time.