Some readers of this blog may recall that several months ago I had wished for a Simple VPN device – back then I had written:
There is another type of software that ought to be moved into its own box, and that is Virtual Private Network (VPN) client and server software. Yes, I’m aware of OpenVPN, and I tried to find setup instructions that someone like me could understand, but to no avail – it looks like you need a degree in computer networking to understand how to set up this type of software. And yet, built into hardware devices, it could be immensely useful in certain circumstances. Let’s consider the following diagram:
In this particular case we have a SIP-based VoIP adapter at a remote location. Anyone who has worked with Asterisk behind the wrong kind of firewall knows the issues involved with using SIP and not having things set up just so (one-way audio, anyone)? But also, we may for whatever reason want that “remote” VoiP adapter to appear as if it were on the local network (maybe we have an ISP playing games with SIP packets?). So we plug the VoIP adapter into our “VPN Client-Side Device” and on the other end, we have a companion “VPN Server-Side Device” which in this case makes two connections to the router – one to receive the “tunneled” data and the second to send the unencrypted data back onto the local network. The green arrows represent the “tunnel”, the orange arrows show where the data from the VoIP adapter enters and exits the tunnel. Please note this is entirely a wired connection, we aren’t using wireless anywhere here. Also note that as far as the VoIP adapter is concerned, the only network it can “see” is the one at the other end of the tunnel – under no circumstances can it access the Internet other than by going through the tunnel.
I show this using a VoIP adapter, but I’m sure that people could think of a lot of other ways this could be used, and a lot of other devices that could be connected to the client end.
Now some will probably argue that it is inefficient to have a device that does nothing but provide the tunnel. But that’s the point – almost anyone could set this up. If you send the client-side device to your grandmother, she can set it up (well, maybe that’s pushing it a bit, but you get my point). People who would never touch a Linux box or a server could use this.
I want to tell you, when I wrote that I had no idea just how difficult it would be to actually get VPN tunneling working using OpenVPN. The problem isn’t that it doesn’t work — actually, it works quite well — the problem is that it’s a real bear to set up, unless you have someone who knows what they are doing walk you through it. Unfortunately, you may not haves someone who knows what they are doing to help you, so you’re stuck with me (unless you can find a better page on the subject – if such exists, please let us know in a comment).
The real issue is that you have to learn so many new things at once to make this work. So, my attempt here is going to be to try and give you a “cookbook”, with plenty of screenshots so you can see how things actually look when everything is configured. I’m actually also going to approach this in a slightly backward manner, showing you how to configure the client first, then the server. My theory is that you will think that configuring the client is so simple that configuring the server can’t be much harder, and you’ll get sucked into the project before you realize what you’ve gotten yourself into! 🙂
By the way, what we are doing here doesn’t look exactly like the diagram above – instead it looks more like this (note that the “Primary Router” on the client side can be omitted if you don’t need any UN-tunneled connections, or if the DSL or Cable modem has a built-in router):
For our client, we need something that you can plug your desired device into. You may have a laptop, and you want to communicate securely with a home office. You may have a VoIP adapter, and you want to make secure calls to a remote Asterisk server. While you can run a software client on the Laptop, when you have a hardware device like a VoIP adapter your choices become more limited. The solution is to purchase a router that is capable of being re-flashed with custom firmware. In this case we are going to use the Tomato firmware, and in particular, a version of the firmware designed to support OpenVPN as either a client or a server. Here we’re going to use it as a client. The idea will be that ANY device plugged into the router will automatically use the VPN tunnel, and if for some reason the tunnel isn’t available then the connected device will not be able to communicate, therefore there’s little chance that an insecure communication can take place.
Some readers will already have a router capable of running Tomato firmware, and some will not. The main Tomato Firmware page shows which routers are supported. If you do not already have one of these and plan to buy one, I recommend that you consider the Asus WL-520GU. I know that the main Tomato firmware page says there’s no USB support for that model, but that’s not necessary for what we’re going to do, and it’s not even true if you use the recommended firmware build. The reason I suggest using the WL-520GU is because I’m told that although it’s not totally impossible to “brick” the router by doing a bad flash, if you do make a mistake, your chances of being able to recover (so that the router isn’t consigned to being an expensive paperweight) are far better than with some other models.
I should mention here that I inherited this project from someone else who couldn’t get it to work. I had made the mistake of casually suggesting they get the Asus router if they wanted to attempt this, only to find them on my doorstep with router in hand. So again, I’m not saying this will be easy, and the one thing I will not tell you how to do is how to get the Tomato firmware onto your router. The reason is that if I give you instructions and leave out a step and you brick your router, you will be mad at me. Better you find someone else’s instructions, and if they’ve left out a step, you can be mad at them. I will mention that, at least in the case of the Asus router I had here, it was a two-step process – I used the Asus Firmware Restoration Utility to first install DD-WRT (using these instructions) and then used DD-WRT’s web interface (Administration -> Firmware Upgrade) to install Tomato (I’m skipping a whole tale of woe and grief that transpired between those two events). Basically I followed the instructions at An Easy Guide to Installing Tomato on the Asus 520gu, but I’m not telling you to do that — it’s up to you which instructions you wish to follow.
Why Tomato instead of DD-WRT? Because Tomato works, that’s why. But if you want to try getting it working on DD-WRT, go ahead, knock yourself silly (only one tip for you, from a tweet by @pista01 on Twitter — if you keep getting TLS errors, make sure your NTP client is set up). If you succeed, great for you. If you don’t, you’re welcome to come back here and continue on.
But don’t just grab the first build of the Tomato firmware that you see. You need one that includes VPN support. There are two versions I would highly recommend — if you have taken my advice and acquired an Asus WL-520GU (or similar model with built-in USB port), then I recommend teddy bear’s build because it enables the “missing” USB support, and also includes the VPN support from SgtPepperKSU’s build — which is the one you should get if you DON’T have the Asus WL-520GU, but instead have some other compatible router (EDIT: Advanced users may also wish to check out thor2002ro’s build, which offers both the VPN and USB support from the aforementioned builds, plus support for SDHC, SNMP, and perhaps other additional features. I haven’t tested that one at all, and note that recent versions probably won’t work with many router models due to memory requirements, so unless you really need one of the features in that version and know that your router supports it, I’d stick with one of the other versions). Make sure you read up on the chosen build and be sure you get the correct firmware version. For the Asus I used the binary from inside the tomato-1.25-ND-USB-8632-vpn3.3.rar archive, but since then a newer version has been released (tomato-1.25-ND-USB-8634-vpn3.4.rar was released in August, 2009), or you may prefer a different version.
In this series you will see several screenshots. I’m using the custom “Tomato USB” theme, so the colors and “look” may be a bit different than what you see, but other than that everything’s in the same place as with the default theme. Later (after taking the screenshots) I replaced the tomato.png file from the theme with the one you see at the right, which I happen to like a little bit better.
Two other caveats: Although this router has wireless capability, for this application we aren’t going to use it — it’s going to be used on a wired network only. That said, once you get it set up and working, feel free to experiment with the wireless capabilities if you like — just be aware that if something on the wireless side of things doesn’t work, I really can’t assist you. And also, we’re assuming that everything plugged into this router will be using the VPN tunnel full time, which implies that this router might (perhaps in a majority of cases) be plugged into another router, so that some other devices can access the local Internet connection, while the devices plugged into this router are limited to going through the tunnel. Note that if this router is plugged into another router, it would be preferable (but not absolutely essential) if it were in that router’s DMZ, so that you don’t have double NAT (Network Address Translation) taking place.
And now, a word about OpenVPN. OpenVPN supports two different modes of operation — TAP and TUN. In this case, we are using TUN. We MIGHT explore TAP at a later time, but TUN is easier to set up, and ANYTHING that can be done to simplify this process is worthwhile. The main difference, from the users point of view, is that using TUN the two ends of the tunnel occupy two different portions of the local address range. In this case, on our “home” LAN, the addresses are in the 192.168.0.x range and are assigned by the main router. At the client end of the tunnel, the router running the Tomato firmware will hand out addresses in the 192.168.5.x range. Neither router steps on the other’s toes, so to speak, when handing out addresses. And there is one other difference — although shared directories can be accessed across the network, Windows/Samba shares cannot be “seen” on opposite ends of the tunnel. If you know they are there and know the IP address and share name of the hosting device, you can still access them, but the mechanism that advertises shares doesn’t cross the tunnel.
With TAP, on the other hand, the router at the primary location hands out IP addresses in the same local address range to devices on both sides of the tunnel, and also (in theory, anyway) shares will be “advertised” across the tunnel. Sound like what you’d want, right? Except that when we tried to set it up on the server side, somehow we managed to bring down the entire network – it basically acted like a denial-of-service attack on the entire LAN, and the problem stopped the moment we went back to using TUN. After you’ve messed with this stuff long enough, something like that can leave a rather bitter taste in your mouth, so we decided to stick with what worked. In our particular application, seeing shares across the LAN would not be essential. Even if you eventually want to try using TAP, I suggest setting it up using TUN first, then when you have achieved that you can cross your fingers and try switching the mode to TUN at both ends. I doubt it will work easily for you, but experimentation is certainly welcome.
If you want to know more about TUN/TAP and other OpenVPN options, you might want to get the book Beginning OpenVPN 2.0.9 (Amazon affiliate link) by Markus Feilner and Norbert Graf — see my recent mini-review of this book (links edited February, 2010 to reflect updated and expanded edition of the book).
So with the preliminaries out of the way, let’s get to the client screenshots…
This first shot shows the header, sidebar, and Save/Cancel buttons. Since the settings are probably a bit hard to read, we’ll zoom in on the pertinent part:
The main thing to keep in mind here is that the subnet you select must not conflict with address assignments on the primary network at the server. Next, on the Router Identification page, enter a Hostname to identify the router:
One thing I have read in several places is that it’s important for the time to be set accurately at the client. For your choice of time servers there are several presets, but if you use Custom you can specify one or more of your own as the first choice(s), if you have a machine on your network that acts as an NTP server:
At this point you should probably go to the Administration | Admin Access page and set things up there to your liking. Remember that anything you make accessible on the router’s local LAN will also be accessible on the other side of your tunnel using the same security methods, so be careful.
Now to what we came for… setting up the VPN client. Click on VPN Tunneling and then on Client. I’ll say that one more time – you MUST click on Client. It’s far too easy to skip that step and accidentally be trying to configure a server! Then you should be at the Client 1, Basic tab:
Of course, the button at the bottom of the page will sat “Start Now” instead of “Stop Now” – I took these screenshots through the tunnel, so I couldn’t very well stop it to get that little detail right! After setting that up you want to click on the Advanced tab:
Note that the connection retry value is -1 (which means infinite retries) and there are two added lines in the custom configuration section:
keepalive 10 120
float
In case the connection to the server or the Internet goes down for a time, those settings should cause the client to keep attempting to re-establish the connection to the server. The float command is useful if your client is at a location where the ISP might change the IP address without advance notice — it is supposed to allow the VPN connection to survive an IP address change. You can omit float if your client is at a fixed IP address that is not subject to change.
And then the Keys tab – just take a look now, when we move on to the server I’ll explain how these are filled in:
There is one more thing that needs to be done (besides adding the keys) for our tunnel on the client side. Go to Administration | Scripts and click on the WAN Up tab. Enter the following two lines into the text box:
route del -net 0.0.0.0
route add -host `nvram get vpn_client1_addr` gw `nvram get wan_gateway`
The second line may wrap on your display due to length. You may want to copy and paste those lines so you get them right, but if you must type them, note that the ` characters are actually backticks – the small apostrophe-like character on the key to the left of the “1” key (the number 1) near the top left corner of most keyboards (well, in my part of the world, anyway). Then click on the Save button way down at the bottom of the page. If you fail to do this and your tunnel ever goes down (server dies or is inaccessible, etc.) there is a possibility that traffic that should go through the tunnel will use the local Internet connection instead. These two lines will keep that from happening. When you are done it should look like this:
Just a note for future reference: You can optionally add an additional line here to allow you to get to something “upstream” of the network on the WAN side of the router. For example, let’s say you have configured the server to allow you to connect to devices on both the LAN subnet (192.168.5.x in our example here) and also devices connected to the primary router, in other words, on the WAN side of the router running the Tomato firmware (which were in the 192.168.1.x range on our example setup). But let’s suppose that upstream of that, you have a cable modem at 192.168.100.1 at the client location that you’d also like to be able to access. As long as you don’t have a conflicting address elsewhere in your network, you could add a line such as this in your WAN Up script:
route add -net 192.168.100.0 netmask 255.255.255.0 gw `nvram get wan_gateway`
… which would allow access to anything in the range of 192.168.100.0 through 192.168.100.255. If you want to be more specific, you can narrow it down to an exact address:
route add -host 192.168.100.1 gw `nvram get wan_gateway`
We’ll cover what else has to be done to enable this type of additional routing at the server end in the upcoming installments, but the above is what has to be done at the client-side router that is running the Tomato firmware. Note that you do not normally have to a a line such as this to get to devices on the same subnets as the LAN or WAN ports of the router, but just for anything upstream of the network that the WAN port of the router is connected to. If you don’t understand why you might want to add this type of additional routing now, just ignore this information for the time being, but make sure you do add the first two lines I mentioned above.
Once your tunnel is operational, you should go to the Advanced | Routing page and look at the Current Routing Table. Your addresses will differ (this was a test setup; you probably will see a wider range of addresses) but the main thing you want to make sure is that there is never an entry of default (or 0.0.0.0) in the destination column that goes to any interface other than a tunnel (tun11 in this illustration) — you should never see that whether the tunnel is enabled or disabled. If you do, then something’s likely wrong with the two lines you entered above (under the WAN Up tab):
So, that’s the basic client setup. Wasn’t too difficult, right? Ah, but just wait until we get going on the server — hopefully I can make it easy enough that you won’t have about three weeks worth of sleepless nights, when you occasionally mutter under your breath things like “Why? Why?? Why??? WHY won’t this damn thing work!”, and other things I’d rather not put on the Internet! But before I can write the next part, I REALLY need to catch some ZZZ’s, so the server will have to wait for part 2.
I am using linksys pap2 cisco device as a telephone adaptor The service provider has blocke the sound how i can use tunnelingor some other solution to cheat the server
with reards
Thanks!
I read that teddy bear’s vpn build leaves no jffs space on a 4mb flash router. Does this mean that each time the router starts the vpn details need to be re-entered?
I’m torn between the WL-520GU and the WL-500G V2 Platinum. I’m intending to run 2-3 OpenVPN tunnels over 2mbit internet, a usb/samba share and a usb printer. Would I be better off going with the higher spec’d router or do you think the WL-520GU would be sufficient?
Thanks!
Adam: With regard to your first question, I’ve never had to re-enter the details. The one caveat is, you have to put in just the minimum required details as indicated in this article (only the BEGIN and END lines, and the part in between). You do NOT include the entire contents of the .crt or .key files.
As for your second question, I’m really the wrong person to ask. I’d go into the Tomato forum and post that question if I were you:
http://www.linksysinfo.org/forums/forumdisplay.php?s=53cb36f11bb66d05920ad035c13acdab&f=160
Please understand that just because I have figured out how to do one particular thing, and I share that with my readers, does not make me an expert on the subject — far from it in this particular case. Getting this VPN tunnel to work was quite possibly the hardest thing I’ve ever done on a computer, and if you’re doing something different than what I did, I have no way of knowing if it will work.
I am just wondering since you’ve emphasized so much on restricting DNS to the remote server instead of using the local DNS server what happens in case of restarting a new connection if server IP has changed because of new ip assignment? Since your client relies on being able to resolve the server host name to reach the host, with a cutoff you could have no access to your sole source of DNS resolution so how on earth are you able to resolve and restart a new connection with the remote server?
chin, what are you talking about? The term “DNS” isn’t even found anywhere on this page, except in these comments!
If you’re talking about something I said on a different page, then please post your comment on that page and include some context so I know what you’re referring to!
@ adam and for others,
I’ve used teddybear’s Tomato VPN on both the Asus RT-N16 and the WL-520GU. Both work well but I wouldn’t use the WL-520GU for a busy VPN. I’ve got 3 VPNs working using RT-N16s at both ends. The RT-N16 is the better choice for a full time VPN.
As far as TAP VPN, I always use it. I’m not an expert but have it working with each side of the tunnel getting DHCP from its own router. They use their own DNS. The internet is not routed through the server.
I used TAP with this setup for 2 reasons, to keep each network online if one side of the VPN goes down and to have network browsing work.
Each side has a small DHCP range, with the servers and printers all having static addresses.
[Editor’s note: The above message was edited for sentence structure, capitalization, and punctuation.]
Nice posting Dude…. keep posting…good show…
I also know why TAP took down your network. The funny thing is though is that I know nothing about OpenVPN and i have never touched it. But saying that, it uses TCP/IP…which I am good at 😉
An MS RRAS server uses similar technology for a PPTP connection when both the local and remote subnets are in the same range.
Maybe an explanation of how LAN communication works first will help…
When a node sees that the destination IP address is on it’s own local subnet it does an ARP request. This is a broadcast out to the network which is sent to EVERY device on the LAN. The broadcast basically says “hello, anyone there? Who has the Ip address 192.168.0.8?”
The device that has this responds with it’s MAC address (which is what the other node needs) like “Hi, it is me 2382627FA”. At this point the two communicate using MAC addresses.
The reason this is important to bring up is because computers on a LAN DO NOT EVER use IP addresses to talk to each other. The point of IP addresses is to determine whether a node is on the local network or a remote network. Before routing was around nodes communicated using MAC addresses and they could never communicate with nodes on a different LAN. This is the point of the router. Also a broadcast (using ARP to get the MAC address) is limited to the LAN.
So think what happens when 192.168.0.1 on the local subnet wants to talk to 192.168.0.5 on the remote subnet. This will fail (or should do). 1 thinks that 5 is on the local subnet (LAN) so it does an ARP broadcast to get the MAC address. The problem though is that as mentioned already a broadcast is limited to the local LAN segment so it isn’t sent over the VPN to the other side and it fails. They can’t communicate. This is why subnets at different sites should be on different IP ranges. This is how normal networking works. We will come back to this in a bit and you will see why it is important…
Have you ever heard of a network loop? It is where you basically have a network cabling setup that goes is in a loop. For example two switches joined with one uplink cable is normal. Imagine a broadcast here. It means a packet enters the switch (from a node) and is sent out EVERY port. It goes down the uplink to the other switch which also sends the broadcast out every port. What is important to note here is that a broadcast is sent out every OTHER port except the one it received it from. If it didn’t then the two switches would keep bouncing back the broadcast between the uplink thus creating a network loop. The packet never dies and as more and more broadcast occur on the network more and more broadcasts bounce between the two and never die. You can imagine the end result…The network gets completely congested and it takes down the network, this is called a broadcast storm. This does not happen though by not repeating the broadcast out the port it received it on. However if you have two cables (or two uplinks) to the same switches then you create a network loop. This concept is important to understand so you will see why your network goes down using TAP. Basically in a network loop broadcasts just “loop” around the network and eventually take it down. In your network I have no doubt there ISN’T a network loop but instead you will have artificially created one by accident explained next.
Going back to my 2nd para about ARP. Of course eventually there was a need using VPN’s for a local subnet to be able to communicate with a remote subnet even if they were on the same IP range. Natively this will never work so something called Proxy ARP was created. It is a very simple concept. When an ARP broadcast is received on a device running proxy arp it is repeated down the other interface. Let’s use the same example above and show how it now works. 192.168.0.1 does an ARP broadcast for 5 which is sent to ALL devices on the LAN including the router. The router receives this but it normally wouldn’t do anything with it because it is not the owner of the IP address. But, running Proxy ARP is repeats the ARP broadcast down the VPN. Of course now 5 receives this and replies with it’s MAC to the router. This now allows communication between both subnets at the LAN level. It acts as a bridge at the layer 2 (MAC address level). This is how it is possible.
Now onto the real issue…at last.
This IS how TAP will function because it wouldn’t work any other way. In this mode it will be using Proxy ARP. What this means is that ARP broadcasts are repeated out the other interface. Why is this a problem? Well if you are not careful you can create a network loop and I bet I know how how your remote site is set up. It is this way (using your diagrams):
VPN server connects in the LAN port of router.
There is at least one other switch/hub device connected another port on the router and this network cabling system eventually goes back to the VPN server on it’s other interface.
This is what happens when you turn TAP on.
All ARP broadcasts the VPN server receives on it’s MAN interface get repeated out the VPN interface to the LAN port on the router. As it is a broadcast the router repeats it again out all interfaces except the one it received on. The switch receives it and does the same. This repeats until eventually it arrives back at the VPN server on the LAN interface THAT DID NOT SEND IT. And then what happens? It uses proxy ARP again to repeat it out the VPN interface to the LAN port again of the router. You have a network loop when you turn this feature on and it takes down your network.
Knowing all this above now you should be able to trace the steps involved to eliminate the loop. You will have to move some cables around.
D.A.R.Y.L., I appreciate the explanation. The only problem is, there are no devices on either network (except routers and switches) that have multiple interfaces, and there are no loops. The basic (intended) hookup was this (tunnel portion in italics):
Internet —> Cable/DSL modem —> Consumer grade hardware router —> Centos Server with only one NIC —> OpenVPN tunnel entrance —> Centos server (out same NIC) —> Consumer grade hardware router —> Cable/DSL modem —> Internet —> Cable/DSL modem —> Consumer grade hardware router (remote end) —> Tomato-firmware based Router —> OpenVPN tunnel exit —> Four LAN ports on Tomato-firmware based Router —> Devices plugged into any of those four ports.
The above works if TUN mode is used but not if TAP mode is used.
I can see where my diagram might have led you to believe that there were two NICs on the Centos server, but there are not. The traffic flows into the centos server and then (encrypted) out the tunnel on the SAME hardware NIC. There are no devices on the entire network (either end) with multiple NICs. And there are no data loops (two port of a router connected to two ports of a switch, etc.) The network isn’t that large — such a thing would definitely be noticed! Having said that, what actually happened when I attempted to use TAP mode sure sounds a lot like your “broadcast storm”, so you may be on the right track, but given what I’ve just said, I don’t understand how it could possibly have happened.
OpenVPN is VERY popular (though NOT particularly easy to set up, at least not in my experience), so I’m a little surprised you haven’t run into it before.
If you remember I said in my explantion that you created an “artificial loop” not a true networking loop. You have said that there are no two cables linking into the same two switches. I also said I knew you would have checked this…as it is obvious. The artificial loop will be by the VPN server being connected on one NIC then somehow
There must be a loop somewhere because enabling TAP would enabled proxy ARP and this takes down the network. Ragardless of this is doesn’t matter anymore. I now know that the IP address ranges DO NOT need to be the same. In fact it is non standard to do so and since this is no longer a requirement we can forget about TAP and just look at the TUN set up.
it seems from the current set up you have now achieved everything execpt forcing all traffic down the VPN even when it isn’t up? When it is not up internet access goes out normally which you do not want. This is easily doable by modifying the routing table on the router. What i am going to explain I tested on a windows machine but the principle should be the same on a router.
The first thing you need to do is find out the public IP address of the VPN server (example 1.1.1.1). Now you create a static route on the router as follows (this is the windows command, translate it to your OpenVPN commands):
Examples:
Routers WAN IP is 2.2.2.2 and the gateway IP (on the WAN side) is 2.2.2.3. Run the command:
route add 1.1.1.1 mask 255.255.255.255 2.2.2.3 interface 2.2.2.2 metric 1
This adds a route stating that to get to the public IP (ONLY) of the VPN server (1.1.1.1.) always go out through the gateway 2.2.2.3 (router’s own default gateway) via the interface 2.2.2.2 (its public WAN IP).
Now edit the TCP/IP settings of the WAN interface of the router. Configure them manually but set the default gateway to a black hole address (an IP that doesn’t exist, maybe a local one). This will screw up the default gateway route. In other words whenever the router needs to get to other networks it fails everytime, therefore block internet access. But because we added an exact static route to the VPN server it will work for traffic to there only. It finds the matching rule in the routing table and sends it out the correct gateway. This will allow the router to establish the VPN tunnel. After that everything will flow across the VPN like before anyway but if it ever goes down there is no internet access for the clients. Does that make sense?
BTW this is requires that you have static public IP addresses. NO OTHER soution will work.
I must be really bad at explaining things today.
Right now the setup IS doing what we want, including limiting traffic from the Tomato-firmware based router to only being able to get to the Internet by going through the OpenVPN server. Somewhere in this series of articles it mentions how I did that, and it’s a very similar technique to what you suggested, adding static routes to the router.
By the way, this works even though both ends are on dynamic IP addresses (although the server end has a DynDNS address, so that it can be found by the client). If the server’s IP address changes then sometimes (perhaps always, it hasn’t happened often enough to establish a pattern) the Tomato router must be rebooted to restore connectivity.
My real question was why I couldn’t get TAP mode to work. TAP mode is supposedly a feature of OpenVPN. If you have no experience with OpenVPN then you are not going to be able to answer that question. I’m not disagreeing with your assessment that there may have been a broadcast storm, BUT if you read about TAP mode you will find that it is supposed to create an extension of the local network at the distant end. To quote Wikipedia (http://en.wikipedia.org/wiki/TUN/TAP):
The advantage of TAP would be that devices at both ends of the tunnel should be able to see and communicate with each other without any routing tricks. Some have said that even Windows/samba shares can be seen across such a network, though obviously I can’t confirm that (it does NOT happen when TUN mode is used).
Again, if you have no experience with OpenVPN at all and you’ve never read up on it, then you may not be aware of the supposed capabilities (and advantages) of TAP mode. It’s not that big a deal that I couldn’t get it to work, but I’ve always wondered why.
You’re right and here are some links I haven’t posted yet. I don’t have any experience of OpenVPN but as I said i don’t need to. I haven’t even looked on the internet about it but what you just described about what TAP does is EXACTLY what proxy arp does. Take a look at it and you will see it bridges two LANs together at the layer 2 level. Sound familiar? TAP does the same.
TAP is using proxy ARP. I didn’t even look this up I just knew it because as I said all of these technologies run using the standardised TCP/IP. I know the limitations of TCP/IP and what it can and can’t do. I don’t need to know how to configure OpenVPN. My aim is to guide you in what TCP/IP can and can’t do. Then YOU type the commands for it in OpenVPN. If you look at the proxy ARP link read the bit about joining two broadcasts domains together. This what is happening with TAP (and Proxy ARP). Now read Network loop. Notice what it says about repeating broadcasts at the layer 2 level? Notice how all 3 technologies I mention all work at layer 2 and join LANs together? And that a loop causes broadcast storms which take down the network? And you experience this when TAP is enabled? Everything is pointing to it being that, the facts are undeniable.
D.A.R.Y.L., I sort of understand what you’re saying (anything related to networking is a bit hard for me to wrap my head around).
But, here is what mystifies me. OpenVPN offers TAP mode. People use TAP mode. If you go to a document called Ethernet Bridging – OpenVPN, it says this:
And further down:
I had, of course, hoped to use the second method, although it wouldn’t have ruined my day if I’d had to use the first.
I know the article mentions that “you have the bridge-utils package installed” and I did that. In fact, in part 3 of the series I talk about how the Webmin module default paths for starting and stopping the bridge, as well as the path to DOWN-ROOT-PLUGIN are wrong and need to be corrected. And in part 4 I discuss what actually happened after I did yum install bridge-utils and then attempted to use TAP mode (it starts just a bit past the halfway point in that article).
My problem is that you talk as if TAP mode can’t possibly work, And if I were going solely by my own personal experience, I might have been inclined to agree with you. And yet… and yet… lots of folks seem to have got it working.
I have no doubt that I missed a step, or that either the Webmin OpenVPN module or I left out some vital configuration option, or something. But at the time, I could could not figure out what that something might be, and it becomes REALLY difficult to try and diagnose a problem when virtually nothing on your local network is reachable. But if you’re saying that it can’t be made to work, then apparently all the folks that say that they have it working are living in some fantasy world. So at the time, and because this whole project had already becaome the most difficult thing I’d ever attempted to do, I settled for using TUN mode. It wasn’t what I wanted, and I have always wondered why (in the context of OpenVPN) I couldn’t get it to work.
And if you know nothing about OpenVPN, you are not going to be able to give me the answer to that.
I appreciate your explanation for why I had the problem in the first place – your “broadcast storm” scenario sounds like exactly what I encountered. And yet, when OpenVPN is configured in TAP mode on a WORKING installation, there must be some mechanism that avoids this. And I have always wondered just what that is, and what I may have missed in attempting to get TAP mode to work. It’s that one unsolved mystery that, while not causing me to lose any sleep, will always be in the back of my mind as something I should have been able to make work, but failed!
I think you are misunderstanding what I am saying. I have never said once that it can’t work. In fact (no offence) but you seem to be changing you questions every time, then when i answer them you talk about something else. IE in that email you sent to me you said you wanted the setup to work with both subnets being on the same address range. I addressed this in a comment here which you then answered telling me that it now works (using different subnets) and that is no longer and issue. So then (as you asked for to see if I could help) I read you comments and email again and the only thing outstanding was to route all traffic across the VPN. So I then wrote details about that. Then you tell me this is already working and change your question again asking me why I think TAP failed. You keep chaning the requirements lol. I know you don’t mean it but look at the email you sent me then look at the comments in order and you will each time I am trying to answer what (i think) you are asking…but then you change it.This last one about TAP – your previous comment you asked why it doens’t work. I answered sainy why I think it isn’t working. You never asked once how to get it to work and I assumed this wasn’t ncasry because on previous comments you said it is now working?!?! You clearly imply from your previous comment that you are just curious why it failed using TAP. Anyway I answered why I think it isn’t working and now you are telling me it does work for other people. Yeah I know this, I never disputed it. However on your setup there is obviously a problem. That problem is that when you try TAP it seems to take down the network. I am telling you what the most likely cause of that is. If you can eliminate or find what is causing this I’m pretty sure the TAP setup will be plain sailing afterwards…
So what exactly are you asking me? 😉
Do you want help setting this up with TAP or just want to understand bits and pieces related to it? Either way, if you have a choice over TAP or TUN you should choose TUN.
Let me tell you soemthing about WordPress.com…it is a joke. I was just logged in and wrote a whopping reply, clicked the post button and it refreshed asking me to login then redirected me back here with nothing I had wrote…fuming!
I’ll keep this one brief.
I think you are misunderstanding me. I have never said once that TAP can’t work. I am trying to address YOUR problem and why it doens’t work for YOU. In fact (no offence) but you ask me a question then when I answer it you move onto something else saying that is not what you meant. From the email you sent me you said you wanted them on the same subnet range. I then asked you if it was a necessary requirement and you said no. So this already contradicts what your email said?!?! You said that the site (now) is up and running using different subnets and all is working. So I re-read your email and the only thing that appears to be outstanding (you want help with) was routing all internet traffic down the VPN but you then said that is also working. The only thing left was why TAP failed, but at no point did you imply you need it working. I addresses this next and now you think I am telling you it won’t work at all. I’m not, there is obvioulsy a problem on your setup when you enable TAP which takes down the network. I am just trying to figure out what this is first; you can’t progress without it. This needs to be eliminated before anything else is even attempted. I am pretty sure that once it is eliminated the TAP will be plain sailing afterwards.
From your questions about TAP they clearly imply that you “are just wondering” why it failed. That is exactly what I addresses but now it appears from your last comment you actually do want it setup this way. You keep changing the goal posts lol! I know you are not doing it on purpose but you need to be a little more clearer to me 😉
What exactly do you want me to help you with? Is it just to know why TAP failed or do you actually want TAP configured? Or are you happy with TUN (which you said is working)?
No comments to this blog are posted until I manually approve them, that is why you don’t see them. I used to allow automatic approval of posts by people who’d previously had a comment approved, but all it takes is one or two jackasses to spoil it for everyone, so now NOTHING gets through without my explicit approval. And I DO sleep occasionally! 🙂
Right, I’ve been thinking about this and a way I can explain it easily to you and I think I have it.
Firstly let’s talk about a switch. A switch is a bridge. If you read up on how bridges and switches work they basically repeat the eletrical signals they receive on one port down another port (the destination port). In the case of broadcasts the data is repeated down ALL ports except the port it was received on. This is to prevent a network look already explained above. Imagine we have two switches joined together via one cable. A broadcast is done on one of them from a PC. This data is flooded down all ports except the one it was received on. One of these ports is where the switch connects to the other switch. The broadcast goes down this port to the other switch and is again repeated down all ports except the one it receive it on. The broadcast packet doesn’t loop. If the switch sent the broadcast back out on ALL ports then it would be sent back to the other switch, then the process repeats again and sent out all ports again..back to the other switch thus a loop occurs. This is why a bridge/switch NEVER repeats a broadcast down the port it received it on; it is the only way to prevent a network loop. This is how it should work.
Now imagine we connect them via two cables ( you know what happens here but maybe not the WHY). switch1 is connected to switch2 using ports 1 and 1 like for like. They are also connected with the other cable ports 2 and 2. A broadast is received on switch1 from a pc on port 3. Switch 1 repeats this down all ports execpt port 3. Switch 2 receives it through port 1 then repeats it down all ports except port 1 (including port 2). Now switch one receives the broadcast on port 2 and repeats it down all ports except port 2. Out port 1 it goes again to switch 2 thus a loop is created. That is a physical loop.
Now imagine we have one switch and one server with two NICs, both plugged into the switch. Nothing else is connected. When NIC1 one does a broadcast the swtich repeats it on the port of that NIC2 is connected to. The PC receives the broadcast on NIC2. What do you think happens here? Nothing actually because the PC isn’t a switch and does repeat a broadcast anywhere. In this case the physical cabling appears to be a loop but a network look does not exist.
Now, in same scenario we bridge our two NICs together. Now the PC does a broadcast it goes out both NIC’s (but we will only follow one of them -NIC1). the switch receives the broadcast from NIC1 and repeats it down the port that NIC2 is attached to. NIC2 is configured as a bridge so repeats it down NIC1. Thus you have a loop again but this time it was articially created.
This is what you have done. And thinking about it a bit more I think I know how. SOMEWHERE and it doesn’t mean it has to be on the VPNserver site I think you have bridged the wrong interfaces. This may have been done on the client site on the tomato router. Because remember if the loop is there it will also travel across the VPN and take out the VPNserver site. This is something I overlooked before.
I can say though that even though you think the cabling is all ok I am 100% certain you have a loop. This is why I said it is an articial one. You may be thinking about it in your head and are telling youself that there is no cables set up in a loop. Yes this is correct but somehow there is a loop most likely through the tomato router or VPN server. If you would have bridged the TAP interface with ETH interface on both devices this should not be an issue. This is why I think you must have bridged the wrong interfaces accidently.
D.A.R.Y.L., you has sent me an e-mail implying that you were something of an expert on networking. I had suggested you look at this series of articles and see if you could figure out where I went wrong. This is exactly what I had wrote to you:
Notice that I did not specifically say in that e-mail what was or was not working; my only allusion to that was “to get it working to the degree that it was when I published the series.” I had assumed you’d read the entire series before commenting, but in your first reply, you said, “That is one long post! i started reading it then stopped. I’ll just ask a few questions first”. You also admitted you know nothing at all about OpenVPN, yet you have proceeded to post as if you completely understand what is happening — and the insinuation is always that I did something wrong and created a loop. But, everything I did is laid out in the articles, and in the first reply to you in this thread (my post submitted on 2011/08/05 at 8:14 am), I show you a diagram of exactly how it’s laid out.
I have explained in subsequent posts (and I apologize if this was not clear to you from the beginning) that the tunnel is working exactly as I wanted to, EXCEPT that I could not get TAP mode to work (and that should have been apparent if you’d read the series). The reasons I wanted TAP mode to work were so that all the addresses on both sides of the tunnel were in the same address range (192.168.0.x), therefore hopefully enabling network shares on either side of the tunnel to be “visible” on the other side – including, hopefully, Samba/Windows networking shares. THAT was the thing I was asking about. I realize now (and, actually, earlier in this thread) that you may have been confused because in my e-mail I said, “Just so you know, one of the primary requirements was that there must not be ANY way, under ANY circumstances, for a device plugged into the Tomato router to access the local network (and therefore go out to the Internet from) the remote location. If the tunnel was down, then devices connected to the LAN ports on the router shouldn’t be able to connect to ANY part of the Internet.” I was only mentioning that to say that this was a primary requirement that can’t be broken by any configuration changes, and to explain the routing used on the Tomato-firmware based router. I did NOT intend to imply that this was not working — indeed it does work that way, and very well.
Once I discovered that you don’t know the first thing about OpenVPN, I realized you’re not going to be able to give me the answer to this problem. And I have tried to say as much. But still you continue suggesting that there has to be a loop somewhere. Well, I’ve described the configuration, so where’s the loop? Again, if you knew OpenVPN you might see an obvious mistake, or know what typically would cause such loops (I can’t be the only person this has ever happened to), but since you don’t know OpenVPN you have no idea what the problem yet, and yet you continue to imply that there was something wrong with the configuration. Which there may well have been, but my bet is that it was in the OpenVPN TAP configuration, not in any physical cabling problems. I will just say again, there are no computers on either side of the tunnel with multiple NIC’s.
In any case it is apparent that we are now going in circles. All that’s happening is you’re getting more annoyed (evident by the tone of your posts) and I’m starting to feel the same, and you really aren’t giving me any solid clues as to what I may have done wrong (because you don’t know OpenVPN). So let’s end this here. I appreciate that you attempted to help to the best of your knowledge, which does not include OpenVPN. I definitely appreciate your explanation of the “broadcast storm”, which confirms the initial suspicion I had about what happened when I tried to use TAP mode. So it’s not as though I (and, hopefully, anyone who has read this far) have not gotten anything out of this discussion, but I think we’ve gone as far as we can go down this path.
No problem. I agree we prob aren’t going to get anywhere with this now. I would consider myself an expert in TCP/IP but not OpenVPN. What I was trying to imply is that once OpenVPN passes the information down to the TCP/IP stack I can help you at that point. And since this is a networking issue that affects the entire network it has to be related TCP/IP. I have 10 years experience of this type of stuff and as you mentioned in the blog post the only thing that takes down a network like this is some kind of DOS which floods the network. To do this DOS on a LAN (inside a protected network) and take out an entire network is only possible using broadcast storms (which is brough on by a network loop). I am sorry if I was a bit pushy saying that it has to be this but in all the 10 years I have been doing this I have NEVER seen anything else be the cause of your exact symptons. Every single time it has been a network loop. So as at this point I can’t help you anymore. Please update it here of course if you ever find the cause as I am curious of this myself 😉
I’d just like to clarify a few points you raised as well.
Just because I don’t know how to configure OpenVPN it doesn’t mean I don’t know about networking. I can tell from your site that you are a telephony enginner. This obviously means you know a lot about VoIP in general and the various protocols like SIP etc. You may know how to configure Asterix, Avaya and Nortel phone systems but maybe not a draytek PBX (for example). Does this mean you are not an expert in VoIP and can’t help someone who has one? If I had that draytek system and knew nothing about VoIP but used SIP as the protocol, without you knowing how to configure the draytek system you could guide me and tell me what things will work and what won’t because it uses SIP. IE you probably know that SIP has issues going over a NAT device and needs STUN to help it. Regardless of what kit you use, if you use SIP as the protocol you are limited by these factors. If you know the problem is related to SIP it wouldn’t matter whether you knew how to configure a draytek would it? You could tell me the issue is relating to SIP and how to fix it. This is exactly what I was doing with your OpenVPN and is what I mean by I don’t need to know how it works. The thing in common with windows PTPP, Open VPN, L2TP and all other VPN protocols is that they are all using TCP/IP and limitied by it’s “rules” and they all tunnel traffic. As you said yourself you don’t know much about networking. For all you know you could have configured the OpenVPN perfectly fine but it is something about the underlying networking technology that is the problem. This is what I helping you with and clarifying.
I do think that was a bit harsh for the reasons I said above. I know for a fact (whether it is a loop or not) I have narrowed the problem down for you. No one else commenting on your posts got that far and I think we both agree it sounds like a broadcast storm. I have a very good idea what the problem is but I can only help so much and have to rely on you for the network topology.
There is something you learn when you start out in networking called the OSI model. It talks about how certain protocols operate at different “layers” in the TCP/IP stack. In these layers they can’t affect other protocols on other layers (kind of, bit complicted to explain properly). A program (like OpenVPN) operates at the highest layer of the OSI model (layer 7, application layer). What this means is that it has to pass through all the other layers before the data gets onto the wire. When it does this it has NOTHING to do with the application anymore, instead it uses the TCP/IP protocols like TCP, PTPP and IP and ARP, NAT etc. These are standardised protocols that do not change no matter what program invokes them. It is at these layers (about half way through the stack) where your issue is. And that is perfectly in my territory. If this same setup was required with a windows VPN server, PTPP server, IPsec VPN server or netgear routers that support this feature EVERY ONE on them would have this same problem. What I’ve been trying to get you to understand (which is where I’ve got frustrated from) is that they all use TCP/IP and the problem is at the level. It isn’t so much OpenVPN but you have been very dismissive of my advice because “I know nothing about OpenVPN”. Yes it because of a config change at the OpenVPN level but the reason I say it doens’t matter is because the same config change on all other types of VPN server would also cause the same issue. In other words the issue what they all have in common is TCP/IP not openVPN… If you ever get to the bottom of this you will realise this.
D.A.R.Y.L., your whole problem is you make a lot of assumptions, and sometimes you couldn’t be more wrong. Such as…
Not even close. What on earth would have given you that idea?
And, I’m not disputing your expertise on TCP/IP, although I personally find much of what you write very hard to understand (it’s one thing to know a subject, and quite another to be able to explain it to others. My eighth grade algebra teacher may have known algebra, but he couldn’t teach a kid how to make a paper airplane, let alone factor a polynomial). But I was just trying to point out that if you don’t know OpenVPN, it’s incredibly presumptuous to assume that it was some kind of network misconfiguration that caused this problem, and not a simple configuration error in setting up OpenVPN itself.
To oversimplify things just a bit, this entire site is for amateurs, by an amateur. You are not explaining what you know at an amateur level. I can overlook that, some people just can’t help themselves from talking “over the heads” of their intended audience, and I think I at least get about 75% of what you’re saying (the previous post being an exception — you threw so much jargon and so many acronyms into that last paragraph that I was totally lost). But when you admittedly don’t know the first thing about OpenVPN, then you should refrain from making any assumptions about what might be causing this issue, because you simply have no way of knowing whether or how OpenVPN itself could be misconfigured to cause this sort of problem.
Oh, and by the way, I STILL don’t know how to factor a polynomial — nor do I have the slightest idea why anyone would want to! 😉
You don’t give yourself the credit you deserve mate. You aren’t bad at networking! That’s why I’ve been quite technical 🙂 and so are some of the people that comment like Jake on another post about routing. The fact you set all this up shows you are ok (at least) at networking and the other article I commented on about isolating the two subnets using a sibnet mask. These are pretty good you know for “someone who doesn’t know much about networking”. I assumed would understand most of what I said. In fact I’ve been told I am very good at bringing things highly technical down to a level others can understand. Of course this is only possible once I know the level someone is at. You can see this by my computer networking basics article I wrote. Just covers the basics and explains what they all do.
Well maybe because of your domain name and the fact that your VoIP, asterix and telephony categories and articles are massive compared to the others lol!
NOTICE: All comments above this one were imported from the original Michigan Telephone Blog and may or may not be relevant to the edited article above.