Saturday 29 June 2013

FortiGate with ZTE MF100 3G USB modem and T-Mobile for a backup Internet connection



Overview


I found an old O2 3G USB  broadband Internet dongle sitting around in a box and decided to put it to good use.  Here is how I've used it to act as a backup for my main wired broadband Internet connection at home.


Prerequisites


You will require the following:
  • Properly functioning broadband Internet connection and LAN/WLAN at home.
  • FortiGate firewall running FortiOS 5.0 (I've used v5.0,build0208 GA Patch 3).
  • A mobile broadband Internet dongle (USB) with a valid SIM (I used a ZTE MF100 from O2).
  • Mobile Internet APN, username and password (for T-Mobile APN=everywhere; username=eesecure; password=secure).
Note, your USB modem may be provider locked, like mine was to O2.  If it is, you will need to unlock it first, by either calling your provider and paying them some of your hard-earned cash, or using this software.

Configuration


Insert your 3G USB dongle with a SIM card into one of the spare USB ports on the FortiGate firewall and reboot it.  If you don't reboot the firewall, the USB modem may not get detected properly.

Custom Modem Definition


To enable modem settings, run the following commands on the console:
 config system modem   
  set status enable   
 end  

Unless your modem is on the supported list, you will need to add a custom entry.  Have a look through the list in the GUI first to see if your modem is listed and if it is, select it:
Network -> Modem -> Configure Modem

Otherwise, identify your modem via a USB bus scan:
 # fnsysctl cat /proc/bus/usb/devices    
 ...    
 T: Bus=01 Lev=02 Prnt=02 Port=01 Cnt=01 Dev#= 5 Spd=480 MxCh= 0    
 D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1    
 P: Vendor=19d2 ProdID=0017 Rev= 0.00    
 S: Manufacturer=ZTE,Incorporated    
 S: Product=ZTE WCDMA Technologies MSM    
 S: SerialNumber=P671A1ZTED010000    
 C:* #Ifs= 5 Cfg#= 1 Atr=e0 MxPw r=500mA    
 I: If#= 0 Alt= 0 #EPs= 2 Cls=ff(vend.) Sub=ff Prot=ff Driver=serial    
 E: Ad=81(I) Atr=02(Bulk) MxPS= 512 Ivl=0ms    
 E: Ad=01(O) Atr=02(Bulk) MxPS= 512 Ivl=4ms    
 ...   

Then, using the Vendor and ProdID from the output above, create a custom modem:
 config system 3g-modem custom   
  edit 1   
   set vendor "ZTE"   
   set model "MF100"   
   set vendor-id 19d2   
   set product-id 0017   
  next   
 end  

Next, you will need to find the serial port used by your modem, because your modem may actually appear on two different serial ports, possibly due to this related bug.

Scan through all available ports on your FortiGate as follows, by issuing a dial command:
 # diag sys modem com /dev/ttyusb0  
 Serial port: /dev/ttyusb0  
 Press Ctrl+W to exit.  
   
 # diag sys modem com /dev/ttyusb1  
 Serial port: /dev/ttyusb1  
 Press Ctrl+W to exit.    
 atdt*99#  
 NO CARRIER  
   
 # diag sys modem com /dev/ttyusb2  
 Serial port: /dev/ttyusb2  
 Press Ctrl+W to exit.   
 atdt*99#  
 CONNECT 3600000  
   
 # diag sys modem com /dev/ttyusb3  
 Serial port: /dev/ttyusb3  
 Press Ctrl+W to exit.  

In this case the port you want is /dev/ttyusb2, since there is a dial-tone.  Unfortunately I am not quite sure how to hang-up the line correctly after this test, so I end up power cycling the firewall at this point.


Modem Configuration


We can now configure our modem:
 config system modem  
   set status enable  
   set mode redundant  
   set dial-on-demand enable  
   set connect-timeout 30  
   set interface "external"  # your primary Internet interface
   set wireless-port 3  
   set phone1 "*99#"  
   set username1 "APN username"  
   set passwd1 "APN password"
   set extra-init1 "AT+CGDCONT=1,\"IP\",\"your.provider.APN\""  
   set distance 100  
 end

Note, see this Fortinet KB article regarding how to map ttyusbX to wireless-port, which is "3" in this case.


Check to make sure the system can tak to your modem now and there is a dial-tone:
 # diag sys modem detect  
 modem is attached.  
 dialtone is detected.  
   
 # diag sys modem query  
 USB status: Connected  
 manufacturer: ZTE INCORPORATED  
 model: MF100  
 IMEI number: XXXX  
 SIM state: Unknown  
 service status: Unknown  
 signal level: 1/4  
 network name: Orange  
 network type: UTRAN  
 location area code:   
 active profile(AT&V):  
 &C: 2; &D: 2; &E: 0; &F: 0; &S: 0; &W: 0; E: 1; L: 0; M: 0; Q: 0; V: 1;  
 X: 1; Z: 0; \Q: 3; \S: 0; \V: 0; S0: 0; S2: 43; S3: 13; S4: 10; S5: 8;  
 S6: 2; S7: 50; S8: 2; S9: 6; S10: 14; S11: 95; S30: 60; S103: 1; S104: 1;  
 +FCLASS: 0; +ICF: 3,3; +IFC: 2,2; +IPR: 115200; +DR: 0; +DS: 0,0,2048,6;  
 +WS46: 12; +CBST: 0,0,1;  
 +CRLP: (61,61,48,6,0),(61,61,48,6,1),(240,240,52,6,2);  
 +CV120: 1,1,1,0,0,0; +CHSN: 0,0,0,0; +CSSN: 0,0; +CREG: 0; +CGREG: 0;  
 +CFUN:; +CSCS: "IRA"; +CSTA: 129; +CR: 0; +CRC: 0; +CMEE: 2; +CGDCONT: (1,"IP","everywhere","0.0.0.0",0,0)  
 ; +CGDSCONT: ; +CGTFT: ; +CGEQREQ: ; +CGEQMIN: ; +CGQREQ: ; +CGQMIN: ;  
 ...


Dead Gateway Detection


You need to re-configure your primary "external" interface to detect a failed interface condition.  Find your next-hop router first:


 $ traceroute -m 2 -d 1.1.1.1  
 traceroute to 1.1.1.1 (1.1.1.1), 2 hops max, 52 byte packets  
  1 192.168.67.254 (192.168.67.254) 282.554 ms 1.091 ms 0.977 ms  
  2 217.33.154.151 (217.33.154.151) 13.737 ms 15.177 ms 324.736 ms  


Then, configure the gateway detect settings on your primary external interface:
 config router gwdetect  
   edit 1  
     set interface "external"  
     set server "217.33.154.151"  
   next  
 end  
    
 config system interface  
   edit "external"  
     set fail-detect enable  
     set fail-detect-option detectserver  
     ...
   next  
 end  


Finally, adjust the modem interface with some important additional parameters:
 config system interface  
   edit "modem"  
     set distance 100  
     set defaultgw enable  
     set dns-server-override disable  
     ...  
   next  
 end  

Firewall Policy


You need to allow access from your internal network to your backup external interface (i.e. modem).  the best way to do this, is to group your primary external interface and modem into an interface zone as follows:
 config system zone  
   edit "external_zone"  
     set interface "external" "modem"  
   next  
 end  


Then, you can update your existing outbound firewall policy to use the zone as the destination interface:
 config firewall policy  
   edit 1  
     set srcintf "internal"  
     set dstintf "external_zone"  
     set srcaddr "net_192.168.67.0_24"  
     set dstaddr "all"  
     set action accept  
     set schedule "always"  
     set service "ALL"   
     set nat enable  
   next  
 end  


Dynamic DNS (Optional)


You may want to configure DDNS on your 3G interface, if you are planning to connect to services on your internal network during fail-over:
 config system ddns  
   edit 1  
     set ddns-server dyndns.org
     set ddns-domain "yourhostname.dyndns.org"  
     set ddns-username "your username"  
     set ddns-password "your password"  
     set monitor-interface "modem"  
   next  
 end  


Testing


While your monitor an external IP with a ping session from a host on your local network, drop your primary external interface and observe the modem interface take over.

Note, if you have a tunnelled IPv6 configuration on your primary interface, be prepared for network instability once you fail-over to the modem interface.  Since there is no way to specify the modem interface on your sit-tunnel, the tunnel is effectively broken for the duration of the fail-over.

References


I've used the following reference material to prepare the solution described in this article.  Many thanks to the respective authors.


Monday 24 June 2013

A practical guide to setting up IPv6, NAT64/DNS64 on a FortiGate firewall (FortiOS 5.0) using a consumer broadband Internet connection



Overview


Even though I am a couple of years too late for World IPv6 Launch, I've finally decided to implement IPv6 at home.  The following step by step guide, should help anyone with access to similar hardware in implementing this solution.







Prerequisites


You will require the following:
The following steps should hopefully guide you in setting up a tunneled, but fully functional IPv4/IPv6 configuration, which will further enable you to switch off IPv4 entirely and use NAT64 to continue accessing IPv4 resources from your IPv6-only network.


Configuration


In the subsequent sections, the following example parameters are used.  Make sure to substitute your own settings:

LAN firewall interface: internal
WAN firewall interface: external
Client IPv6 Address: 2001:470:1234:567::2/64
Routed /64 subnet: 2001:470:890a:bcd::/64
LAN IPv6 interface IP: 2001:470:890a:bcd::1/64
DHCP6 scope: 2001:470:890a:bcd::1000/112

IPv6 Tunnel


The first step is to establish a tunnel to your IPv6 provider, which in this case will be tunnelbroker.net.

First, allow tunnelbroker.net ICMP (ping) access to your IPv4 public IP:
 config system accprofile  
   edit "no_access"  
   next  
 end  
 config system admin  
   edit "HE"  
     set trusthost1 66.220.2.74 255.255.255.255  
     set accprofile "no_access"  
     set vdom "root"  
     set password "your_own_secret"  
   next  
 end  

Also make sure your external interface allows ping access:
 config system interface  
  # your external interface name may be different  
  edit "external"  
   set allowaccess ping ...   
  next   
 end   

Next, create a regular tunnel following this link.  Once you've created the tunnel, configure your firewall as follows:

 config system sit-tunnel  
   edit "HE"  
     set destination 216.66.80.26  
     set interface "external"
     set ip6 "client IPv6 address/mask from HE portal (e.g. 2001:470:1234:567::2/64)"
   next  
 end  
 config router static6  
   edit 1  
     set device "HE"  
   next  
 end  

Test your work so far, by pinging your server's IPv6 tunnel end-point IP:
 # exec ping6 2001:470:1234:567::1  
 PING 2001:470:1234:567::1(2001:470:1234:567::1) 56 data bytes  
 64 bytes from 2001:470:1234:567::1: icmp_seq=1 ttl=64 time=13.4 ms  
 64 bytes from 2001:470:1234:567::1: icmp_seq=2 ttl=64 time=13.2 ms  
 ...


Also try pinging an external IPv6 IP (e.g. Google):
 # exec ping6 ipv6.google.com  
 PING ipv6.google.com(2a00:1450:4009:802::1012) 56 data bytes  
 64 bytes from 2a00:1450:4009:802::1012: icmp_seq=1 ttl=59 time=14.3 ms  
 64 bytes from 2a00:1450:4009:802::1012: icmp_seq=2 ttl=59 time=14.0 ms  
 ...  


Public DNS


If you have a dynamic IPv4 IP assigned to you by your Internet service provider, you would probably want a DNS name automatically updated, when it changes.

Using the HE portal, on your Tunnel Details page, click "Edit" next to "rDNS delegation" and click the link titled "Delegate to dns.he.net".

Next, head to dns.he.net and add a new domain, which you have registered with an IPv6 compliant registrar (check with the registra if they support "IPv6 glue records").


Edit the newly added domain zone, create a new A host record (e.g. myip.mydomain.com), making sure to tick "Enable entry for dynamic dns" and set the TTL to 5 minutes.


Click the DDNS icon next to you new record and generate a new API key.


Got to the "Advanced" tab of the tunnelbroker.net tunnel management portal and register your hostname and API key.


Last, go to your domain name registrar's admin portal and delegate your entire domain or a sub-domain to HE.net's name servers (there are five).



Dynamic DNS (DDNS)



Since I do not have a static IP address with my Internet service provider, I've configured FortiOS to update my external IPv4 address, when it changes using dyn.com as follows:
 config system ddns   
  edit 1   
   set ddns-server dyndns.org   
   set ddns-domain "myhost.dyndns.org"   
   set ddns-username "my username"   
   set ddns-password "my password"   
   set monitor-interface "external"   
  next   
 end   

The problem with this approach however, is that your IPv6 tunnel provider doesn't know about those changes and cannot update your tunnel's public IPv4 address.  So, every time you reboot your DSL/cable modem and get a new IP, your IPv6 tunnel will be down.

FotiOS currently only supports the following DDNS services:
 FortiGuardDDNS FortiGuard DDNS service.   
 dhs.org        members.dhs.org   
 dipdns.net     dipdnsserver.dipdns.com   
 dyndns.org     members.dyndns.org and dnsalias.com   
 dyns.net       www.dyns.net   
 easydns.com    members.easydns.com   
 genericDDNS    Generic DDNS based on RFC2136.   
 now.net.cn     ip.todayisp.com   
 ods.org        ods.org   
 tzo.com        rh.tzo.com   
 vavic.com      Peanut Hull   

Note, HE.net is not on the list and since there is no generic dyndns2 protocol option, there is no way to tell your FortiGate firewall to automatically update your tunnel end-point IP.

So a workaround I've put in place, is to configure ddclient according to these instructions, to run on a local Linux server, periodically detect my public IP and update the tunnel end-point if it changes.


For reference, my /etc/ddclient.conf looks like this:
 # Configuration file for ddclient   
 #   
 # /etc/ddclient.conf   
    
 daemon=600                # check every 600 seconds   
 syslog=yes                # log update msgs to syslog   
 #mail=root                # mail all msgs to root   
 #mail-failure=root        # mail failed update msgs to root   
 pid=/var/run/ddclient.pid # record PID in file.   
 ssl=yes                   # use ssl-support. Works with   
                           # ssl-library   
   
 protocol=dyndns2   
 use=web, web=checkip.dyndns.org, web-skip='IP Address'   
 server=ipv4.tunnelbroker.net   
 script=/nic/update   
 login=your_username  
 password=your_password  
 username=#.tunnel.tserv#.lon#.ipv6.he.net    

This process updates your tunnel end-point IPv4 and since you've configured a hostname and API key on the tunnelbroker.net portal previously, it will also perform a chained update to your HE.net DDNS host entry.


In effect you will have two separate DDNS entries which both point to my dynamic IPv4 address:

  • myip.mydomain.com
  • myhost.dyndns.org


Ideally, you would want your firewall/router to do this as part of it's PPPoE interface-up routines, but we are where we are..


EDIT: I've written a very basic script to obtain the current public IP from the FortiGate firewall directly, rather than going out to a remote site.

To use it, save it to your favourite script location and update the "use=" line in your ddclient.conf as follows:
 ...
 use=cmd, cmd=/scripts/use-fg.sh  
 ...


Internal LAN


Now, configure your LAN interface(s) to support IPv6:
 config system interface   
  # your internal interface name may be different   
  edit "internal"   
   ...   
   config ipv6    
   set ip6-allowaccess ping https ssh snmp    
   set ip6-address "first IPv6/mask in the routed/64 prefix from HE portal (e.g. 2001:470:890a:bcd::1/64)"   
   set ip6-send-adv enable    
   set ip6-manage-flag enable    
   set ip6-other-flag enable    
    config ip6-prefix-list    
    edit "routed/64 IPv6 prefix from HE portal e.g. 2001:470:890a:bcd::/64"    
     set autonomous-flag disable  
     set onlink-flag enable    
    next    
    end    
   end    
  next    
 end   

Some notes on important flags:

  • ip6-send-adv enables router advertisement messages.
  • autonomous-flag enables stateless IPv6 configuration (dynamically generated IPv6 addresses within the prefix).
  • ip6-manage-flag means that there is a DHCP6 server on the network handing out IPs (stateful).
  • ip6-other-flag means the DHCP6 server is also handing out DNS information, etc.
  • onlink-flag basically means the prefix is on a local (layer 2) network.


Note, since we are using a stateful configuration where DHCP6 hands out IPv6 addresses, we disable the autonomous-flag to stop a second stateless IPv6 address being assigned.

Configure IPv6 address objects, which you will use later on in your IPv6 firewall policies:
 config firewall address6  
   edit "all"  
   next  
   edit "net_2001:470:890a:bcd::/64"  
     set ip6 2001:470:890a:bcd::/64  
   next  
 end  

Configure IPv6 firewall policies to allow access from your internal network and also to allow ICMP (ping) from hosts on the internet:
 config firewall policy6  
   edit 1  
     set srcintf "HE"  
     set dstintf "internal"  
     set srcaddr "all"  
     set dstaddr "net_2001:470:890a:bcd::/64"  
     set action accept  
     set schedule "always"  
     set service "ALL_ICMP6"  
   next  
   edit 2  
     set srcintf "internal"  
     set dstintf "HE"  
     set srcaddr "net_2001:470:890a:bcd::/64"  
     set dstaddr "all"  
     set action accept  
     set schedule "always"  
     set service "ALL"  
   next  
 end     

Test you work so far, by pinging your LAN IPv6 IP using an online ping tool while watching the tunnel interface with the packet sniffer:
 # diag sniffer packet HE "icmp6" 4   
 interfaces=[HE]   
 filters=[icmp6]   
 pcap_lookupnet: HE: no IPv4 address assigned   
 4.211481 HE -- 2001:1640:3::3 -> 2001:470:890a:bcd::1: icmp6: echo request seq 1   
 4.211575 HE -- 2001:470:890a:bcd::1 -> 2001:1640:3::3: icmp6: echo reply seq 1   
 ...  


DHCP6/DNS


We will be using DHCP6 to hand out IPv6 IPs as well as DNS server information.

Configure local DNS server on the internal firewall interface, which will be handed out by your DHCP server(s):
 config system dns-server  
   edit "internal"  
   next  
 end  

Configure DHCP6 server:
 config system dhcp6 server  
   edit 1  
     set interface "internal"  
       config ip-range  
         edit 1  
           set end-ip 2001:470:890a:bcd::ffff
           set start-ip 2001:470:890a:bcd::1000
         next  
       end  
     set lease-time 3600  
     set rapid-commit enable  
     set subnet 2001:470:890a:bcd::/112
     set dns-server1 2001:470:890a:bcd::1  
   next  
 end  

Assuming you are also running DHCP on your IPv4 network, you may also want to update it to use the local DNS server:
 config system dhcp server  
   edit 1  
     set dns-service local  
   next  
 end  

Renew your DHCP lease on your IPv6 client and check to make sure it obtained a correct IPv6 address from your DHCP6 server:
 $ ifconfig  
 eth0  Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx   
    inet6 addr: 2001:470:890a:bcd::1000/128 Scope:Global   
    inet6 addr: fe80::ba27:ebff:fee9:d775/64 Scope:Link   
    ...   

Note, you will also see a link-local IPv6 address, which is randomly generated.

Next, make sure the local DNS server on the firewall is handed out by your DHCP6 server:

 $ grep 2001 /etc/resolv.conf  
 nameserver 2001:470:890a:bcd::1     

You can also check the leases on the firewall, to see what IPs your IPv6 clients have been issued:
 BeastGate # exec dhcp6 lease-list  
 Interface  DUID                     IAID   IP                         Expiry  
 internal:xx:xx:xx:xx:xx:xx          15     2001:470:890a:bcd::1000    Mon Jun 24 10:48:27 2013  

Now you should be able to ping an external IPv6 IP from your IPv6 enabled client:
 $ ping6 -c 2 ipv6.google.com  
 PING6(56=40+8+8 bytes) 2001:470:890a:bcd::1000 --> 2a00:1450:4009:809::1012  
 16 bytes from 2a00:1450:4009:809::1012, icmp_seq=0 hlim=58 time=22.427 ms  
 16 bytes from 2a00:1450:4009:809::1012, icmp_seq=1 hlim=58 time=31.112 ms  

NAT64/DNS64 (Optional)


NAT64 is used in pure IPv6 networks to allow access to IPv4 resources.  In practical terms, it means you can switch off IPv4 on your internal network and continue accessing IPv4 sites on the Internet via your IPv6 tunnel.  For compatibility reasons however, you would probably want to operate a mixed IPv4/IPv6 environment, until at least your Internet provider is able to support IPv6 natively.

First, re-configure your system settings to use DNS resolvers other than FortiGuard (e.g. Google and HE.net):
 config system dns  
   set primary 8.8.8.8  
   set secondary 8.8.4.4  
   set ip6-primary 2001:470:20::2  
   set ip6-secondary 2001:4860:4860::8888  
 end  

I found, that when the internal FortiGate DNS server is forwarding to the default FortiGuard DNS resolvers upstream, it completely breaks NAT64.


Next, enable and configure NAT64:
 config system nat64  
   set status enable  
   set always-synthesize-aaaa-record disable  
 end  

Note, we are disabing the automatic synthesis of IPv6 addresses, since we dont want a synthetic address generated if a host already has an IPv6 address.  In this case, an address will by synthethised only if the host does not have an IPv6 AAAA record.

Add an IP pool abd firewall policy to support NAT(ing) of IPv6 addresses to IPv4:

config firewall ippool
    edit "nat64-exit-pool"
    next
end

config firewall policy64  
   edit 1  
     set srcintf "internal"  
     set dstintf "external"  
     set srcaddr "net_2001:470:890a:bcd::/64"  
     set dstaddr "all"  
     set action accept  
     set schedule "always"  
     set service "ALL"
     set ippool enable
     set poolname "nat64-exit-pool"
   next  
 end

Now, switch off IPv4 support on your client and make sure you can no longer access an IPv4 only site (e.g. Fortinet):
 $ curl -I -4 www.fortinet.com  
 curl: (7) Failed to connect to 66.171.121.34: No route to host  

This is expected, since we have turned off IPv4, but told cURL to specifically use it to access the web site.

Next, try to resolve the IPv4 only site using an external DNS server (e.g. Google):
 $ dig aaaa www.fortinet.com @8.8.8.8  
 ...
  
 ;; QUESTION SECTION:  
 ;www.fortinet.com.          IN     AAAA  

Note that there are no IPv6 host records (AAAA) returned for www.fortinet.com.


Now try to resolve the same name using your internal DNS server, running on the firewall, which is now DNS64 enabled:



 $ dig aaaa www.fortinet.com  
 ...  
   
 ;; QUESTION SECTION:  
 ;www.fortinet.com.          IN     AAAA  
   
 ;; ANSWER SECTION:  
 www.fortinet.com.     2339     IN     AAAA     64:ff9b::42ab:7922  
 ...

Note an IPv6 address is synthesised from the NAT64 Well-Known Prefix (64:ff9b::/96).


Finally, check to make sure you can access the web site using IPv6:
 $ curl -I -6 www.fortinet.com  
 HTTP/1.1 200 OK  
 Date: Sun, 23 Jun 2013 17:24:26 GMT  
 Server: Apache/2.2.3 (Red Hat)  
 ...  

Testing


To test your IPv6 setup, head to test-ipv6.com, which should get you a similar result:


Note, at the point you should probably re-enable IPv4 and re-run the test.  The only difference will be, that it won't warn you anymore about limited IPv4 connectivity and the use of NAT64/DNS64.

I hope you found this guide useful and if you find any mistakes or have any suggestions  please feel free to comment.