It has always interested me how things come in waves, I haven’t talked to people about their internal and partner multisite WAN designs in a while, and these past two weeks, it has been the top of everyone’s mind. Sometimes they are concerned with what their partners have access to, others what their remote sites have access to. Every business has a different need. Last week, I discussed the Internet design, and in some spaces this is a dovetail with WAN design, as DMVPN and traditional IPSEC VPN come in through the Internet typically. Some clients want firewalls between everything, others don’t care what the WAN does to other members of the WAN, as long as they don’t hit HQ. But at the same time, HQ needed to provide services to the outlying locations and in some cases, partners. I will talk in this article about WAN design, including what types of WANs to use, and how to architect them to do what you want.
Over the past several years, frame relay has been on a death march and MPLS and Metro E point to point/multipoints on the way up. In typical deployments of MPLS and MetroE multipoint solutions, you lose something major that frame-relay allowed you to have, and that was control over the site to site communications. It used to be that if SITE A wanted to talk to SITE B, all the traffic would come back to HQ, hit a hub router, which, in many cases would have a sub-interface for each site, get turned back around and go to the destination. This allowed for you to put in access-lists or zone based firewall rule sets between these different sites.
Many customers I have worked with have a need to provide a set of services to their other sites, but those other sites operate as independent entities (think of a large conglomerate and you run the network for the holding company). Each of those entities has their own security policy, own networking and support desk teams. You can’t control what antivirus they have or if they even have one, but each one needs to get access to your accounting servers and you are also providing voice services to them.
These nightmares are reality for many customers and most customers who have the problem regarding partners or subsidiaries that need access to some services, also have the first problem of multiple sites of their own. How do you deal with all of this?
Let’s start by defining the primary technologies from a high level.
Metro E: This provides layer 2 connectivity between two or more sites. TYPICAL implementations have all of this traffic going directly between the sites and the ISP has a full mesh of psuedowires between each customer site. There are many ways that this can be done on the ISP side, but that is what the customer sees.
MPLS: MPLS is a method where all of the sites are separated by the ISP layer 3 network, with possibly many hops in between sites. Each site can TYPICALLY see the routes to all of the other sites through the ISP network and go through the ISP directly to the other site.
For most people these are fine options, they have no need to protect sites from each other. Others need the extra control.
For those that need the extra control, one option is to work with the ISP, in the Metro E world, they can provision the tunnels to do a hub and spoke by not creating a full mesh. They can also setup rules between the psuedowires, which only allow traffic to go that is permitted. The downside, is that the ISPs are in control of that, not you.
For MPLS, the same option exists. By manipulating the RD and RT all traffic can be forced back to a hub router. But typically they do not come in through several interfaces. The ISP would probably be unwilling to make changes that block the traffic as it could affect all customers.
Both of the above are not great options. Better option, put your own router (preferred) or firewall at the edge of each remote site facing the ISP. You need to make sure your device works with BGP if you are doing MPLS, unless the provider is controlling your routing (not recommended). Manage your firewalls to block the traffic you don’t want at the edge. This does require a bit of operational overhead, but what security solution doesn’t? There is an added benefit, QOS in the WAN is much better on a router. This is changing, but typically router QOS beats switch QOS any day of the week. I will talk about WAN QOS next week. If you do the router, also make sure to use the firewall feature sets.
Now that you have secured your WAN from itself at the remote sites, how do you deal with HQ which may have 2, 3 or more connectivity methods for WAN? First define what is in your WAN, are VPN users in your WAN? How about vendors and business to business relationships? For each one of the relationships you go back to your operational issues discussed last week of different SLAs, so separate out the failure domains.
For users, consider a VPN concentrator/firewall. Most of the good VPN Firewalls have the full firewall feature set built in that allows you to dictate what that VPN user/group is allowed to do based on who they are. This may require some additional authentication solutions or maybe it integrates directly with AD, either way, differentiate your VPN users and allow them access to only what they need. Put your Internet based remote sites on an IPSEC or DMVPN solution. I like the DMVPN solution because I can setup remote sites with dynamic IP addresses if that is all those sites can afford (I prefer static IP as it provides me operational control with a known endpoint). DMVPN is also a lot easier to setup as once you have it running, you can add more remote sites without doing anything to the head end. On the DMVPN router you can setup the DMVPN tunnel interface as another zone in the zone based firewall configuration.
You can also have all of this backup to an firewall and an IPS before coming into your core. Mix and match all of the options as your security policy needs, if you don’t need sites to be protected from each other, don’t use the firewall features, if you are going to put a firewall in the central site, don’t use the firewall feature set on the DMVPN head end router. I have put a diagram below.
Companies that need to host services:
For the companies that need to host services for groups outside of their control, the above solution may work fine. Just put in access-lists etc. which prevent access to unnecessary resources. Some companies need more though, they need to allow for overlapping IP Space as well as ensuring everyone’s network stays separate…now you are looking at running your own MPLS.
Most companies outsource their MPLS to a service provider, they get their own vRF and that is all they need, but others have a need to control this a bit more, allowing for this virtualization to permeate their own core without putting everyone’s routing table on the core as you don’t know what they are going to do with their own IP addressing and you can’t have overlaps. In this case you may run a Carriers carrier sort of MPLS solution, where an ISP sets up the MPLS, puts each of your subsidiaries on their on vRF and brings them all back to a router on your premise. You setup either run an MPLS with the carrier router (carriers carrier) or your do back to back vRFs where you pull a trunk interface between the ISP router and yours and each VLAN tag gets put into a separate vRF where you begin your internal MPLS process, this is good for smaller solutions, with a few different companies, running this with several often changing vRFs would be difficult operationally.. You then go to a shared vRF with a firewall. This firewall is responsible for NATing connections in separate contexts, the NATted IP addresses are globally unique. The servers in the shared vRF respond to the NATted address and the firewall picks it up and unNATs and sends it back through the correct vRF.
For companies who may have both remote offices they own and those which are B2B relationships, I recommend looking at splitting the WAN. Work with your provider to bring the B2B relationships in over a separate WAN circuit so you can provide that group with a different level of restriction.
Share this Post