4 :revdate: {sys:date "+%Y-%m-%d %H:%M:%S"}
10 rrqnet - Packet tunneling over UDP, multiple channels
14 *rrqnet* [ OPTIONS ] _port_ ( _remote_ [*-i* _mac_]* )*
18 *rrqnet* is a bi-directional networking plug that channels Ethernet
19 packets between a UDP port and either a tap interface or standard
20 input/output. It is configured on the command line by declarations of
21 the remotes it may communicate with.
26 Note that any options must be given in the fixed order:
28 [-v] [-tpg] [-4] [-B n] [-T n] [-m mcast] [-t tap]
32 This tells *rrqnet* to log its operation on +stderr+. Use *-vv* to
33 also see logs about connections and messaging, or *-vvv* for *rrqnet*
34 to be insanely verbose on +stderr+ about virtually everything.
38 This enables a special UDP transport quirk where *rrqnet* expands any
39 packet in the range [1461,1477] bytes to be of 1478 bytes instead. It
40 is a work-around for an observed problem in a NAT traversal case for
41 certain packet sizes in that range.
45 This directs *rrqnet* to use an ipv4-only socket for its UDP. By
46 default it opens a dual ipv4/ipv6 socket and internally it then uses
47 address mapping for ipv4 (i.e. the ::ffff/96 prefix).
51 This sets the number of receive buffers *rrqnet* should use. The
52 default is computed to be twice the number of dispatch threads (see
53 *-T* below). Receive buffers are pre-allocated and recycled.
57 This sets the number of dispatch threads *rrqnet* should use. The
58 default is 5. The (additional) main thread handles packet reception,
59 where it immediately puts received packets into the buffer queue which
60 is serviced by the dispatch threads for optional decryption, dispatch
61 decision, optional encryption and delivery.
65 This tells *rrqnet* to open an ipv4 UDP multicast channel as an
66 additional remote channel.
70 This tells *rrqnet* to open the nominated tap (the _tap_ interface
71 name) as local channel.
73 * When a tap is used, stdin and stdout are closed, but stderr remains
76 * With "*-*" as tap name, *rrqnet* will use stdin/stdout as local
77 networking channel in a format compatible with VDE plugs.
79 * Without a *-t* argument, *rrqnet* will operate merely as a virtual
80 cable switch among its channels.
82 _address-block[:port][=cryptfile]_ [ *-i* _mac_[,_mac_]* ]::
84 Remotes are declared as +ipv4+ or +ipv6+ network address blocks
85 optionally with port and transport encryption key file pathname, and
86 optionally with specific MAC addresses (in a comma separated list) to
87 ignore. Note that an ipv6 address block might need surrounding square
88 brackets, to avoid confusion with the port number.
93 The intended use of *rrqnet* is to provide VPN (virtual private
94 network) connectivity between hosts. Each VPN host runs its own
95 *rrqnet* daemon to channel the traffic to/from tap interfaces on the
96 hosts via UDP messaging between the hosts.
98 *rrqnet* is prepared for almost any network layout, even including a
99 collection of fully connected hosts, although the more common is a
100 "star' formation. See the EXAMPLES section for inspiration.
102 *rrqnet* includes logic to protect against broadcast cycles. Whilst
103 acting as a switch for all its remotes it does not have spanning tree
104 logic as is offered by bridge interfaces. Generally it's probably best
105 to avoid cabling cycles, but *rrqnet* also handles full mesh linking
106 between multiple hosts without additional configuration. Such a setup
107 is also dynamically adapting to prefer the fastest link between hosts.
109 By default *rrqnet* opens an +ipv6+ socket on the given port. This
110 mode handles both +ipv6+ and +ipv4+ remotes with +ipv4+ remotes
111 handled by means of standard ipv6-mapped address translations. If *-4*
112 is given, *rrqnet* opens a plain +ipv4+ socket instead, and it cannot
113 then have +ipv6+ remotes.
115 An *rrqnet* daemon delivers the packets received from the local end,
116 i.e., the _tap_ or _stdio_, to known remote ends according the
117 targeted MAC addresses through its established remote channels.
118 Likewise, packets from remotes are delivered to the local end or to
119 other remotes according to the targeted MAC addresses. If a packet is
120 an Ethernet broadcast, it is delivered to all (known) channels except
121 the one it came from.
123 If *rrqnet* is started without *-t* option it will operate like an
124 Ethernet switch that provides connectivity among its +UDP+ channels
125 without involving the host network other than for the tunneling.
133 This format declares remotes by +ipv4+ address, with optional network
134 prefix length (0-32), optional port (1-65535) and/or optional key file
135 pathname. *rrqnet* will accept packets from sources that match by
136 prefix. If the network prefix length, +n+, is omitted or given as 32,
137 and a port is given, then it declares an _uplink_.
139 .matching ipv4 uplink and downlink
142 [1.2.3.4]# rrqnet -t vpn0 2300 5.6.7.1:2300=/sec/vpn0.key
143 [5.6.7.1]# rrqnet -t vpn0 2300 1.2.3.0/24:2300=/sec/vpn0.key
147 .plain ipv6 address block
149 This format declares remotes by ipv6 address, with optional network
150 prefix length (0-128) and/or optional key file pathname. *rrqnet* will
151 accept packets from sources that match by prefix. This format (without
152 square brackets) is without port number part, and it thus only
153 declares a prefix mask for allowed remote hosts.
155 .ipv6 address block within square brackets
157 This format declares remotes by ipv6 address, with optional network
158 prefix length (0-128) within square brackets, then optionally a port
159 number and/or an optional key file pathname. *rrqnet* will accept
160 packets from sources that match by prefix. If the network prefix
161 length, +n+, is omitted, or given as 128, and a port number is given,
162 then it declares an _uplink_.
164 .matching ipv6 uplink and downlink
167 [fd::4]# rrqnet -t vpn0 2300 '[fe::1:4]:2300=/sec/vpn0.key'
168 [fe::1:4]# rrqnet -t vpn0 2300 '[fd::/120]:2300=/sec/vpn0.key'
172 The remote declarations define by prefix match the allowed source IP
173 addresses for incoming UDP traffic. A full network address and port
174 (e.g +[fe::1:4]:2300+ of example 2) declares an _uplink_ that the
175 *rrqnet* daemon will establish and maintain by means of regular
176 "heartbeat" messaging. Other declarations define the mask for allowed
177 incoming connections, aka _downlinks_, that the dameon expects the
178 remote end to maintain as uplinks by the remote *rrqnet* daemons.
180 Two (or more) +rrqnet+ plugs may be set up with uplink connections to
181 each other in which case both (all) of them maintain the link.
183 .example of multiple pair-wise uplinks
186 [1.0.0.1]# rrqnet -t vpn0 2300 2.0.0.1:2300 3.0.0.1:2300 4.0.0.1:2300
187 [2.0.0.1]# rrqnet -t vpn0 2300 1.0.0.1:2300 3.0.0.1:2300 4.0.0.1:2300
188 [3.0.0.1]# rrqnet -t vpn0 2300 2.0.0.1:2300 1.0.0.1:2300 4.0.0.1:2300
189 [4.0.0.1]# rrqnet -t vpn0 2300 2.0.0.1:2300 3.0.0.1:2300 1.0.0.1:2300
193 The *-i* option, if used for a remote declaration, is followed by a
194 comma separated list of the MAC addresses to ignore on the associated
195 channel. The *rrqnet* daemon will then just drop any packet with those
196 MAC addresses (whether source or destination) on the remotes of the
202 With the *-m* option, the *rrqnet* daemon also listens for packets on
203 the declared ipv4 multicast address. It is then treated as a separate,
204 persistent remote channel.
206 A multicast channel is declared using the format:
207 *ipv4:port{=keyfile}*. I.e. it includes the multicast ipv4 address,
208 the port number (1-65535), and optionally a key file pathname. The
209 multicast channel is an additional communication channel, but anything
210 received on it will be treated as being from the multicast IP rather
211 than the actual source IP.
213 .multicast example without other remotes
216 # rrqnet -m 244.0.2.1:2000 -t vpn0
220 The multicast channel is compatible with QEMU multicast socket.
225 Transport encryption is added to a channel by means of using a shared
226 key. This is a sizable file, say 1 Mb, of binary data that is used for
227 scrambling the network packets. For example, 1 Mb random data is fine.
229 .preparing 1 Mb key file with random content
232 dd if=/dev/random of=/sec/keyfile bs=1M count=1
236 The key file needs to be copied to all hosts.
238 A channel that has a key file is declared by appending it to the
239 channel declaration, as in the following example.
241 .another example of a downlink with encryption
244 [10.0.0.1]# rrqnet -v -t tap0 1400 10.2.0.0/16:1400=/sec/keyfile
248 That declaration says that all channels with hosts of ipv4 address
249 +10.2.0.0/16+, port 1400, use the file +/sec/keyfile+ for transport
250 encryption. This may also be used as way of restricting remote
251 connections to those that use the right key.
256 Simple rrqnet set up (ipv6)
257 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
258 This is an example of connecting two hosts with *rrqnet* without
259 transport encryption. We pretend these hosts are mutually reachable
260 with ipv6 addresses +fe::2+ and +fe::1:3+ respectively, and we want to
261 use the ipv6 network +fd::1000:0/120+ over *rrqnet*. A nominal set up
262 might then be as follows:
264 .simple rrqnet set up (ipv6)
267 [fe::2]# ip tuntap add tap0 mode tap
268 [fe::2]# ifconfig tap0 fd::1000:10/120 up
269 [fe::2]# rrqnet -v -t tap0 1400 '[fe::1:3]:1400' &
271 [fe::1:3]# ip tuntap add tap0 mode tap
272 [fe::1:3]# ifconfig tap0 fd::1000:20/120 up
273 [fe::1:3]# rrqnet -v -t tap0 1400 '[fe::2]:1400' &
277 Thus, the host +fe::2+ is set up with a tap +tap0+ having ipv6 address
278 +fd::1000:10/120+, and an *rrqnet* daemon for the tap using UDP port
279 +1400+ and an uplink +fe::1:3+ port +1400+. Similarly, the host
280 +fe::1:3+ is set up with a tap +tap0+ having ipv6 address
281 +fd::1000:20/120+, and an *rrqnet* daemon for the tap using UDP port
282 +1400+ and an uplink (back) to +fe::2+ port +1400+.
284 This example also needs ipv6 address resolution set up, which uses the
285 MAC addresses of the two taps. Let's say it's +02:00:00:00:00:02+ for
286 +tap0+ on +fe::2+ and +04:00:00:00:00:04+ for +tap0+ on +fe::1:3+.
287 Then address resolution is established with the following:
289 .example of ipv6 address resolution set up
292 [fe::2]# ip neigh add fd::1000:20 dev tap0 lladdr 04:00:00:00:00:04
294 [fe::1:3]# ip neigh add fd::1000:10 dev tap0 lladdr 02:00:00:00:00:02
298 Simple rrqnet set up (ipv4)
299 ~~~~~~~~~~~~~~~~~~~~~~~~~~~
301 This is an example of connecting two hosts with *rrqnet* without
302 transport encryption. We pretend these hosts are mutually reachable
303 with ipv4 addresses +10.0.0.1+ and +192.168.0.1+ respectively, and we
304 want to use the ipv4 network +10.100.100.0/24+ over *rrqnet*. A
305 nominal set up might be as follows:
307 .Simple rrqnet set up (ipv4)
310 [10.0.0.1]# ip tuntap add tap0 mode tap
311 [10.0.0.1]# ifconfig tap0 10.100.100.1/24 up
312 [10.0.0.1]# rrqnet -v -t tap0 1400 192.168.0.1:1400 &
314 [192.168.0.1]# ip tuntap add tap0 mode tap
315 [192.168.0.1]# ifconfig tap0 10.100.100.2/24 up
316 [192.168.0.1]# rrqnet -v -t tap0 1400 10.0.0.1:1400 &
320 Thus, the host +10.0.0.1+ is set up with a tap +tap0+ having ipv4
321 address +10.100.100.1/24+, and an *rrqnet* daemon for the tap and UDP
322 port +1400+ uplink +192.168.0.1+ port +1400+. Similarly, the host
323 +192.168.0.1+ is set up with a tap +tap0+ having ipv4 address
324 +10.100.100.2/24+, and an *rrqnet* daemon for the tap and UDP port
325 +1400+ and uplink (back) to +10.0.0.1+ port 1400.
327 The kernel automagically performs ipv4 address resolution to learn the
328 MAC addresses associated with the remote ipv4 addresses through the
331 rrqnet set up through NAT
332 ~~~~~~~~~~~~~~~~~~~~~~~~~
334 If one of the hosts, say +192.168.0.1+, is behind a NAT router with
335 different IP, say a dynamic IP on the net +10.2.0.0/16+, we need a
336 different set up. In this scenario, the first host would be set up as
337 a "server" and the second a client that would utilize the router's NAT
338 function for return traffic. The set up would be as follows:
340 .rrqnet set up through NAT
343 [10.0.0.1]# ip tuntap add tap0 mode tap
344 [10.0.0.1]# ifconfig tap0 10.100.100.1 up
345 [10.0.0.1]# rrqnet -v -t tap0 1400 10.2.0.0/16:1400 &
347 [192.168.0.1]# ip tuntap add tap0 mode tap
348 [192.168.0.1]# ifconfig tap0 10.100.100.2 up
349 [192.168.0.1]# rrqnet -v -t tap0 1400 10.0.0.1:1400 &
353 Thus, the "server" is set up to allow connections from any host on the
354 network +10.2.0.0/16+ port 1400 while the "client" is set up the same
355 way as in the simple example above. The client will establish and
356 uphold the connection by virtue of its 30 second "heart beat", and
357 return traffic will be channeled via the router's NAT function.
359 Note that the server sees the _external_ IP of the client and not its
360 _internal_ IP. The server's *rrqnet* therefore has a remote
361 declaration to allow messages from that external IP. In the example
362 case it allows the address block of the 16 bit common prefix (the
363 choice of a 16 bit prefix is merely for the sake of this example).
365 Multiple client hosts
366 ~~~~~~~~~~~~~~~~~~~~~
368 In a "client-server" set up there can be any number of "client" hosts.
369 However, the "clients" behind a common NAT router must then use
370 distinct ports as otherwise the router will be confused about the
373 With multiple remote channels, an *rrqnet* daemon serves as a network
374 switch that forwards traffic in between the channels as well as to and
375 from the "server" tap. The daemon also forwards Ethernet broadcasts
376 out on all established channels in support of ARP messaging.
378 Further, an *rrqnet* daemon may be both a "server" with down-link
379 channels, and a "client" with one or more up-link channels, all at the
380 same time. Such a daemon forwards traffic between all established
381 channels by means of the Ethernet addresses, as well as broadcasts
386 The *rrqnet* daemon may be configured to use standard input/output
387 rather than a tap for local network traffic. This operation mode has
388 more rare use cases, such as linking two *rrqnet* daemons or
389 connecting to a VDE network. For example:
391 .stdio network between two rrqnet plugs
394 # dpipe rrqnet 1400 0.0.0.0/0=keyfile0 = rrqnet 1401 0.0.0.0/0=keyfile1 &
398 The example above connects the two "server" +rrqnet+ daemons operating
399 at different UDP ports, each accepting messages from any ipv4 host.
400 Port +1400+ has +keyfile0+ for transport encryption, and +1401+ has
401 +keyfile1+ for transport encryption.
403 The follwoing, different, example connects the *rrqnet* traffic to a
404 VDE network via a +vde_plug+:
406 .stdio network to a vde_plug
409 # dpipe rrqnet 1400 0.0.0.0/0 = vde_plug /tmp/vde1.ctl &
413 Note that *rrqnet* and +vde_plug+ use compatible stdio packet
419 The UDP receiver in *rrqnet* accepts packets from the specified remote
420 ends only, but it doesn't perform any payload verification. Messages
421 smaller than 12 bytes are taken as "heartbeats", and larger messages
422 are first decrypted as applicable, then treated as Ethernet messages
423 and delivered according to their destination MAC addresses.
425 *rrqnet* bridges its connections, and forwards Ethernet broadcasts to
426 all known end-points except the incoming one. The input logic takes
427 care of avoiding broadcast cycles.
429 *rrqnet* does not have Spanning Tree Logic (STL), but some simple
430 timing logic based on binding MAC addresses to remotes. That binding
431 is sticky for a short time: 6s for broadcast and 20s for unicast. Any
432 packet received during that time from the same MAC address via another
433 remote is dropped. Also, downlinks without incoming traffic for 3
434 minutes are considered stale.
436 *rrqnet* sends a "heartbeat" of an empty UDP message on its uplinks
437 every 30 seconds. This is done in order to maintain the channel with
438 the remote end without actually binding any MAC address.
440 When the local input is a tap, *rrqnet* closes standard input and
441 standard output, but not standard error, before entering the packet
444 Using +-t -+ for stdin/stdout packet traffic is compatible with
449 *rrqnet-cron(8)* - Management script to uphold a *rrqnet* plug.
450 *rrqnet-ifupdown(8)* - Management script to uphold a *rrqnet* plug.
452 *vde_plug(1)* - Virtual Distributed Ethernet plug.
456 Ralph Rönnquist <ralph.ronnquist@gmail.com>