Is there a F/LOSS alternative to Cisco network routers?
Mick Gregg, 2007-05-03
ContentsTerms of ReferenceExecutive SummaryIntroductionHardwareSoftwareConfigurationConclusionEnd NotesTerms of ReferenceThis report makes up the final assessment for my CPIT BCCS391 course, Contemporary Issues in ICT. It has been prepared for Dr Mike Lance at CPIT, being due 3 May 2007. The research I've done has been to answer the question of whether Free/Libre Open Source Software (F/LOSS) provides products that are able to equally compete with Cisco, the network device software and hardware manufacturer and market leader in network routers. The scope has been limited to a particular New Zealand market segment occupied by what I deem to be SME central LAN routers. I have not tried to prove that Cisco or F/LOSS offerings in this segment are good or bad, but have tried to assess whether and how they are comparable, working from the assumption that competition is good for innovation and price, and provides IT decision makers with choice. Part of the comparison is in the relative difficulty of configuration for administrators and of learning administrative skills on the different systems.
Executive SummaryBy comparing the Cisco 2821, Vyatta F/LOSS-based appliance and Quagga project, and how they could be used as a central router in a New Zealand SME LAN, I attempted to answer whether there exists today a real F/LOSS alternative to Cisco in the network router market. After reviewing the abilities of the three options, interviewing experts and analysing third-party reports, I suggest that the Vyatta appliance is a directly comparable router appliance, and deserves to be assessed against, the Cisco 2821. Further, I believe that the open source routing projects I looked into provide readily available and free tools to learn network routing skills with, and that skills learned on either Cisco IOS, Vyatta OFR or Quagga are largely transferable to each other.
IntroductionOne of the most important threads of 21st Century computing has been the rise of Free/Libre Open Source Software (F/LOSS) - described by the GNU General Public License
1 from the Free Software Foundation, for example - which provides for the freedom of end users and programmers to dissect, alter and redistribute software according to their own needs and interests. The condition of this freedom is that they must in turn release their own modifications to GPL software under the GPL if they want to redistribute it.
With major commercial backing from companies like IBM and HP, F/LOSS lead by Linux provides strong and real competition to UNIX and Windows. While still a comparatively small player in the enterprise LAN server market, Linux does compete well with Windows and provides a much needed alternative in a sphere dominated by a single commercial entity. IT decision makers have a real option to consider and good IT departments should be able to use this to their advantage.
This is all well and good for servers, but a major expense in many company networks, and across the Internet as a whole, is in the high quality routers that make large LANs, WANs and inter-networking possible. Cisco (
http://cisco.com ) got in very early in this market, became the best known name as a result and probably enjoys some sales that, as with any market leader, come from the buyer's comfort in choosing a leading brand. Also in line with common commercial practices, Cisco, having such a well known name, uses its name to add a premium to its prices. The question is, does the product that attracts such a brand loyalty have the quality that its price implies?
I have a real, personal interest in this question. I've successfully replaced many Windows file and print servers with Samba, setup VPNs with Openswan, and developed and hosted enterprise web applications in PHP and PostgreSQL, all on Linux-based operating systems. Despite this, I've never had the courage to go past a Cisco router in the middle of my LAN, simply because I've never been sure what the comparative quality of the F/LOSS alternatives are, and have instead trusted the blackbox of a Cisco router. As far as I've been aware, the Cisco router is good at its job, has the WAN interfaces I've wanted and has been recommended to me by the experts, so I've consequently released large chunks of my IT budget for my own peace of mind. Now, in glorious hindsight, I'll find out whether I was a wise purchaser or paid dollars, Euros and Pounds for my ignorance.
Because of this personal interest, and because I necessarily needed to limit the scope of my research into what is a very broad product range, I chose to focus on what I consider to be a typical requirement for a New Zealand SME; the central, main router in its LAN. Somewhere in my typical Kiwi SME's infrastructure, locked away in an air-conditioned server room or sweating in a dusty cupboard, sits this busy wee routing device, segmenting the LAN, connecting to the outside world and working to send network traffic to where it should be going. Many small LANs don't need more than a server with multiple Ethernet cards and a routing daemon that speaks RIP2 to do this job, or even just an ADSL firewall/router to connect to the Internet. I'm going to ignore these very simple scenarios and focus on a larger LAN where there's a need for more sophistication (like a possible requirement for OSPF and VRRP routing protocols), because this is where performance and cost start to become more important, and the comfort of having a Cisco router begins to look attractive to network architects.
HardwareFor this reason I've chosen a Cisco 2821 Integrated Services Router
2 as my baseline. The 2821 setup that we want for our Kiwi SME LAN includes,
- 256MB RAM upgraded to 512MB
- 2 x integrated 10/100/1000 Ethernet port
- an additional 1 x FastEthernet 10/100 port
Cisco doesn't make this too easy to price online, but it does point us to CDW (
http://www.cdw.com ) where we can, if we know what we want, configure and buy our Cisco 2821 router. The base hardware plus the RAM upgrade and additional LAN port comes in at around US$5300, including the standard software and 64MB of Flash memory. Aindriú Ó hEithir, a networking contractor from Dublin and qualified Cisco engineer, tells me (personal communication, 22 March 2007) that the 2821 is not "price competitive" compared with other commercial routers, but believes that Cisco's "legendary" reliability is the reason why SMEs will opt for it. He also cites the availability and relatively cheap cost of Cisco-literate administrators as a factor in Cisco's favour.
Cisco itself didn't reply to my request for comment, but a commercial F/LOSS competitor did. I approached Vyatta (
http://vyatta.com) for an interview about its router appliance, which it directly pitches against Cisco in the corporate LAN. As an open source contributor, Vyatta develops and releases its router software under the GPL and BSD licenses
3, and as part of its commercial operation pre-installs its Debian-based distro on a Dell PowerEdge server, selling this bundle as a commercially supported router. Dave Roberts, the Vice-President, Strategy and Marketing for Vyatta Incorporated, was very happy to have his appliance compared to the Cisco 2821 and told me (personal communication, 7 March 2007) to expect, in some cases, double the performance at half the price. To buy a Vyatta appliance to match the 2821 hardware specification above (from the Vyatta online store) comes in at US$2500, so he seems right about the price, at least.
According to Roberts, a server using a PCI Express chipset is required to better 400 Mbit/s. This is the reason for the Dell PowerEdge, with the bottom-of-the-line one being all that's required. I'm not going to join x86 architecture debate (the one where even Apple has joined Intel), but Vyatta contends that sticking within the x86 "ecosystem" brings all the advantages of choice in using commodity hardware.
Cisco's hardware catalogue is nothing if not exhaustive. It can be difficult to find a person to explain exactly what each offering from Cisco does and which of the many Cisco routers is right for a job. Using the Cisco website to find what you need is even less likely to help you choose. Aindriú Ó hEithir describes it as "incredibly hard to find the right product" (Ó hEithir, 2007). My own experience is that a trusted Cisco reseller's salesman comes up with some model numbers that a Cisco-qualified installation engineer double checks, and we all hope that a better or cheaper configuration wasn't available as the purchase order is signed. The only thing we can be sure of is that at some point we'll find a new feature that's only available to be retrofitted to a different, more expensive, chassis. Opinions vary as to whether the breadth of Cisco's hardware range is a purely commercial attempt to sell new boxes - the expansion card and chassis combinations are "rigidly segment[ed]", to quote Dave Roberts (Roberts, 2007) - a form of lock-in, where the great need to intimately learn a product range breeds brand loyalty, or if this relatively old company has failed to rationalise its product line to the point where now even the experts are confused.
I should be very careful here not to criticise Cisco in isolation. It's hardly the only IT company to baffle buyers with bombast. How many editions of Windows Vista are there and which of Dell's dozen or so notebooks (with three of four sub-varieties each) would run it to perfection, when they all sound so good? Indeed, Vyatta's single choice of a base appliance will surely increase many times as it targets different levels of the router market. Whether anyone else ever reaches the Cisco extent of product range without loosing customers to confusion will be an interesting point to monitor.
My immediate interest lies in the Cisco 2821 and the Vyatta PE860 appliance, so the baffling array of Cisco options can be mostly ignored for the moment. In both cases the buyer gets a computer with a CPU, RAM and interface cards, but in the case of Cisco, he really pays for it. A 1GB RAM upgrade for US$5000 seems very hard to justify, no matter what the quality, and hundreds more again over the Vyatta offering for Ethernet and E3 interfaces seems to be another strangely high cost for the same thing. The 2821 has room to add a 24-port switch module to its chassis (turning it into a switch as well as a router), which is something that Vyatta's appliance simply can't do. It's also something that our typical Kiwi SME probably doesn't want to do, when an external 48-port switch is cheap and the cost and relative advantage in my comparison would weigh heavily against Cisco.
By a stroke of luck, a recent report by the Tolly Group
4 (the date of which, 9 March 2007, is hopefully enough to confirm the coincidence of me researching this paper) compares performances of the two. Though the report was commissioned by Vyatta, the Tolly Group is a commonly-used third party tester and my intention here is to determine whether the F/LOSS options are viable enough to be considered, not to determine exact performance qualities. Cisco's website publishes the results of a test it commissioned from Current Labs
5 in 2004 in which the 2821 throughput is rated as similar to the Tolly Group's results for the Vyatta PE860 (but better than the Tolly Group results for the 2821). In the absence of any other performance data, and without the resources to commission or carry out my own testing, I used the results of the Tolly Group report as a guide, tempered by the Current Labs good results for the 2821. What it shows is that Vyatta's appliance is certainly comparable to Cisco's 2821 and probably superior in performance, for a substantially lower purchase price.
Performance is always going to be dependent on both software and hardware. Sluggish software may be fine on exceptional hardware and a slow processor may be enough for very a clean and lean operating system. Despite this, it seems unlikely that the Cisco IOS software (discussed below) is so inferior to the Vyatta Linux distribution, and the Cisco hardware so superior to the Dell server hardware, that the performance results between the two balance. If we assume that performance is at least equal and that software has a roughly equal affect between the two contenders, then Cisco hardware doesn't seem to be any faster than the entry-level Dell server. In fact, if the Tolly Group report is taken at face value, the Vyatta appliance, routing 64 bit frames, is nearly twice as good as the Cisco 2821, as Dave Roberts predicted. This paints an even more average picture of Cisco hardware, especially considering its price difference.
What appears here is a picture of Cisco selling commodity-quality hardware, but with an extensive array of network interfaces. Though the range of products is large enough to be confusing, and not all pieces work together in a single device, you can build a Cisco router that has exactly the hardware you need. This contrasts strongly with the Vyatta appliance, where hardware is manufactured by partner companies or purchased off the shelf and, for full commercial support at least, is limited to the most usual Ethernet, T1/E1 and T3/E3 interfaces. For our typical Kiwi SME, this limitation may not be an issue, in that we are probably connecting via Ethernet to an ISP-managed router or cheap ADSL modem and don't need any other type of interface. While Yvatta tells me that it will be adding to the interface range over time, today Cisco has a real head start and a F/LOSS option may not exist in the case of an exceptional hardware requirement.
If the hardware interfaces available for Linux servers meet your requirements, there is no comparative hardware quality argument to support the favour that Cisco has. It's time to look at the software to see if this is where Cisco has an edge.
SoftwareCisco IOS is at the core of the company's product range. William Yeagar's multi-protocol router software was licensed by Cisco in 1987 to form the basis of its new Internetwork Operating System. At that time, before the world standardised on TCP/IP, Cisco used its new OS to cement its name in the router market as an enabler that could join together networks of disparate protocols. 20 years later, it's a fair debate as to whether Cisco IOS is a mature and stable system or an aged relic in need of retirement. Its popularity should temper any calls for deprecation, but Cisco's move to a modular replacement (IOS-XR)
6 from a new code-base on its higher end routers provides some evidence that the shared memory and lack of pre-emptive multitasking of its monolithic architecture are of concern. A single badly behaved process in IOS can down the device, and the contrasting claims of lack of Cisco reliability from the F/LOSS community show at least some theoretical foundation here. Also, an update in IOS means a full system update, not just a bug-fix for a single module, so, again, a bad one is system-wide.
While the Linux kernel may be monolithic, the GNU/Linux operating system that Vyatta uses can hardly be described as dated or unstable. XORP, with its professed aims of extensibility and use by equipment vendors and application writers
7, was chosen as the basis of the Vyatta OFR routing software. If the OFR process, or some other process on the Linux system gets out of control, it's usually able to be stabilised again without taking down the whole machine. XORP, and therefore OFR, share some syntax commonality with Juniper's JunOS and could be considered part of this superset, so when you login to the Vyatta terminal, instead of the bash shell that Linux distros commonly use, you'll see the xorpsh shell, accepting its JunOS-like commands.
Vyatta also offers a software-only commercial subscription service and an unsupported, free-to-download version of its Debian-based distro. Using these, it's possible to build your own router using the (x86) hardware of your choice. It's appropriate to consider such a non-commercial, custom appliance, and to broaden the research, I also introduced another player in the F/LOSS routing software community. Zebra (
http://www.zebra.org ) has been around since the late 1990s as a competitor to Cisco IOS. In recent times it was forked as Quagga (
http://quagga.net ), which is currently a more active development project. Its hardware considerations are going to be the same as Vyatta's and the quality and cost of hardware can be decided based on the performance required. Where the non-commercial option will be cheaper than a Vyatta subscription is when no support contract is bought.
Linux is one of a collection of *nix OS options to host Quagga, so fans of Solaris or the BSDs can choose them to build their routers. Quagga itself is modular, with each routing protocol being delivered by a separate daemon and these being abstracted from the kernel by a core Zebra daemon
8. Modules of the suite can be individually upgraded, limiting the effects of a bad upgrade. If its syntax seems to mimic Cisco IOS, its separation of routing protocols into their own daemons and flexibility of base operating system make it a unique offering.
At this level of router there are some expected features. Cisco's 2821 supports the RIP, OSPF and BGP routing protocols, VRRP for high availability, and Ethernet, ATM and PPP encapsulation. It can also act as a DHCP server or relay, a VPN server and a NATed firewall. Vyatta includes all of this in its distro as does the combination of Quagga and whichever Linux distro you choose to run it on. Cisco doesn't make it easy to confirm exactly which of these features are included in the base IOS that comes bundled with the hardware from CDW, so we may or may not need to allow another US$700-US$1200 for feature packs and upgrades. Really, the costs have become irrelevant when the Cisco 2821 is quite clearly at least twice the price of the nearest F/LOSS alternative, but it's important to note that there is no software feature in this set that Cisco offers above the competition. Indeed it's debatable how much our Kiwi SME cares about OSPF, BGP, VRRP or ATM, and whether it would use the central router as a DHCP server.
Extensibility is always touted as an advantage of F/LOSS and Vyatta proved this advantage by adding VPN functionality to OFR, beyond the capability of XORP. Users wanting to extend Cisco IOS can only look back to Cisco to add a feature at the company's discretion. Linux routers can also easily make use of other software in the F/LOSS world and commonly employ tcpdump (
http://www.tcpdump.org ) or Ethereal/Wireshark (
http://www.wireshark.org ) for network monitoring. Vyatta also specifically integrates with the Asterisk Voice over IP server, which compares with Cisco offering the (expensive, of course) addition of CallManager software and its own IP hardware phones.
Another open source concept is that having many eyes on the code helps to identify and fix software bugs, and F/LOSS advocates sing the praises of their software's consequent stability and security. Cisco fans may also believe in the stability and security of IOS, but this may be debated by Michael Lynn, formally of Internet Security System (ISS) (
http://iss.net ). Lynn discovered a major security flaw in IOS and, against orders from his management, made it public. Cisco's reaction was to take Lynn to court
9, leading to a settlement that gags the researcher. In the F/LOSS world, testing and debugging are welcomed.
ConfigurationA common complaint, even from Aindriú Ó hEithir, our de facto Cisco spokesman, is that Cisco IOS GUI tools are underdeveloped and limited. Aindriú actually describes the IOS GUI as "infantile" (Ó hEithir, 2007), saying that, "It is very easy to create a configuration using the CLI that the GUI will interpret as invalid." For this reason, and to acknowledge the command line snobbery that many network administrators might be accused of, I focussed on the CLI tools that the various routing software options offer. If nothing else, this allowed a like-like comparison.
Of course, as with so many systems, expertise in router administration comes mostly from a firm understanding of the theory. Speaking IOS or JunOS or OFR is worthless if I don't know what a router is meant to do and a cookbook administrator may only create a successful configuration by luck or mimicry. Here network administrators may be better equipped than server sysadmins. Vyatta's Dave Roberts does believe that Cisco veers from the standards that the Internet is built on, but doesn't believe that this deviation is enough to be a real problem. Maybe operating at the bottom three layers of the OSI model requires more interoperability, or maybe Cisco doesn't have just enough dominance to shut out the opposition, as other market giants have been accused of, but either way, TCP/IP networking theory can be equally applied across these systems and knowing this theory is a major requirement of configuring them.
I've already uttered the taboo that some of these systems are deferring to a de facto standard and I could even hint that JunOS, and by extension XORP and OFR, though certainly differing in their syntax, aren't a million miles away from Cisco IOS in administrative commands. The simple fact seems to be that Cisco, using Wiiliam Yeagar's multi-protocol router as its starting position, popularised a style of syntax that others have since stayed within reach of. This shouldn't necessarily be criticised. The GNU tools that make up a large chunk of the GNU/Linux operating systems started life as conscious open source rewrites of the tools that UNIX users required in their daily work, and the Linux kernel itself was an attempt to run UNIX at home. As Linux is UNIX-like, in the administrative interface at least, Quagga is IOS-like, OFR is JunOS-like and really JunOS itself is IOS-like. Whether they're siblings, cousins or bastard children isn't an issue, but that the skills applicable to one are transferable to the other is, and it belies any notion that non-Cisco skills would be too hard to introduce to a previously homogeneous environment.
By way a simple demonstration, let's look at how we would login to a Cisco router and set the IP address of its first Ethernet interface to 192.168.0.1/24.
Password:
secrectpasswordcisco> enable
Password:
supersecretpasswordcisco# config
cisco#(config) interface ethernet 0/0
cisco#(config-if) ip address 192.168.0.1 255.255.255.0
Now for the Vyatta OFR way.
vyatta login: vyattauser
Password:
secrectpasswordvyattauser@vyatta> configure
vyattauser@vyatta# edit interfaces ethernet eth0
[edit interfaces ethernet eth0]
vyattauser@vyatta# set address 192.168.0.1 prefix-length 24
You can see that the syntax is different, but you can also see the similarities. Note the enable/configure mode, the > and # prompts to show which mode we're in and the (config)/[edit] hints to tell us what we're configuring. Knowing what an Ethernet interface is and how it should be configured for TCP/IP is the real knowledge required here.
For the sake of it, let's also look at the Quagga way to see how closely its syntax resembles IOS.
Password:
secrectpasswordquagga> enable
Password:
supersecretpasswordquagga# configure terminal
quagga#(config) interface eth0
quagga#(config-if) ip address 192.168.0.1/24
This minor piece of configuration should not be seen to argue that administrators of one system will immediately be able to configure another, but it does highlight that, once a new system is learned, there's no reason to expect initial configurations or regular maintenance to be any more arduous in one system than another. A quick Google search will deliver hordes of interested parties publishing configuration guides and even conversion tables to explain the syntax differences.
10 11 12 13All three options allow ASCII text configuration files to be exported for backup or to use on another system. With Cisco, it's tftp to another machine and with the Linux boxes, whatever system you would normally use for backup. The Linux options offer some real advantages here. Not only is transfer of the configuration between devices very simple, but a lower quality commodity computer using a matching configuration could be made available as a swap-out router for very little expense.
ConclusionIt seems obvious to me that while Cisco does produce a quality device in the 2821 Integrated Services Router, it does so at an expense that is hard to justify when it's compared to the commercially-supported Vyatta PE860 appliance, or a stock *nix server using Quagga. The arguments for using Cisco in this scenario seem to be non-technical requirements for the comfort of the status quo. Aindriú Ó hEithir talks about a need for a revolution to challenge the dominance of Cisco (not that he advocates such a change) but this revolution seems to be needed in the minds of network administrators and architects more than in the technology available.
My own question is answered: I probably have wasted IT budget on Cisco routers. I've had requirements for WAN interfaces outside the Vyatta-supported range - SDSL, for example - but a small amount of time and money testing the unsupported alternatives may have solved those needs, as could outboard modems or alternative WAN connections. Any of these alternatives would have fallen well within the cost of simply buying the Cisco router with its expansion cards and software updates.
I also discovered that I need to do some work improving my own technical skills in router administration. This won't require me to buy a Cisco router or even a Cisco simulator. I think I'll install a couple of new Linux virtual servers, one with Vyatta's free distro and the other using Quagga. It won't cost me a penny and I'll improve with a couple of syntax variations that will be useful even back in the Cisco world. Come the revolution, I'll be ready.
End Notes 1.
GNU General Public License (2 May 2005). Retrieved 8 March 2007 from <
http://www.fsf.org/licensing/licenses/gpl.html >
2.
Cisco 2821 Integrated Services Router. Retrieved 8 March 2007 from <
http://www.cisco.com/en/US/products/ps5880/index.html >
3. Robert Bays (16 January 2007).
OFR License. Retrieved on 1 May 2007 from <
http://vyatta.com/twiki/bin/view/Community/OfrLicense >
4.
Vyatta 1.1.2, Competitive Gigabit Ethernet LAN Routing Throughput Evaluation versus Cisco 2821 Integrated Services Router (8 March 2007). Retrieved 9 March 2007 from <
http://www.tolly.com/DocDetail.aspx?DocNumber=207190 >
5.
Current Analysis Lab Challenge: Cisco 2821 Integrated Services Router (2004). Retrieved 29 March 2007 from <
http://www.cisco.com/application/pdf/en/us/guest/products/ps5880/c1031/cdccont_0900aecd80425258.pdf >
6.
Cisco IOS XR Software Release 3.0. Retrieved 29 March 2007 from <
http://www.cisco.com/en/US/products/ps5845/products_data_sheet09186a008022d5f4.html >
7.
The XORP Vision (16 October 2006). Retrieved 11 April 2007 from <
http://xorp.org/xorp_vision.html >
8.
System Architecture. Retrieved on March 29 2007 from <
http://www.zebra.org/zebra/System-Architecture.html >
9.
Cisco Systems Inc. and Internet Security Systems Inc. v Michael Lynn and Black Hat Inc., 03-CV-03043 (N.D. California, July 27, 2005).
Explanation: Cisco and Internet Security Systems (ISS) filed against Black Hat and Michael Lynn, alleging copyright infringement, trade secret misappropriation and breach of contract. Lynn, an ISS employee at the time, had shown a presentation at the Black Hat conference in Las Vegas that demonstrated how to exploit an IPv6 flaw in Cisco IOS to gain control of a Cisco router. The Plaintiffs were granted an injunction and all parties came to an out of court settlement.
10.
Vyatta & Cisco Commands (24 April 2007). Retrieved on 27 April 2007 from <
http://www.openmaniak.com/vyatta_compare.php >
11.
Cisco/Juniper Commands (26 October 2003). Retrieved on 27 April 2007 from <
http://networking.ringofsaturn.com/Cisco/ciscojuniper.php >
12. Peter Lunqvist, Peter Moyer, and Annette Kay Donnell (2001).
Translations between JUNOS Internet Software and IOS. Retrieved 0n 27 April 2007 from <
http://www.juniper.net/solutions/literature/app_note/350005.pdf >
13. Dominique Cimafranca and Rex Young, IBM developerWorks
(8 October 2003).
Build a network router on Linux. Retrieved on 27 April 2007 from <
http://www-128.ibm.com/developerworks/linux/library/l-emu/ >
