Showing posts with label ip. Show all posts
Showing posts with label ip. Show all posts

Wednesday, 17 May 2017

Virgin Media business cable and static IPs

Here is an interesting problem, if you have a Virgin Media cable (Not fibre or leased line) connection (for business) and have requested the static IP service, initially their sales will try and put you off the static IP asking if you need it and saying there are some issues with the service, but won't really tell you why!

Well unfortunately I've now discovered the issue, so thought I'd post about it for others to be wary of this "business class" service. Firstly, we know how the VM cable network works, they have co-ax to the premises that take you back to their cabinet, from there they mux it back to their central exchange. Now the observant of you will note, no separate cabinet for home or business cable, no separate links to the exchange for home or business, so at all times you are sharing this portion of the network. Not a huge issue as capacity is normally good, but something to be weary of that during 'home' hours (after 5pm generally) the service will be noticeably slower.
So this now bring the interesting problem to light, you are connecting to a home service with business features (static IP, etc).


Virgin Media uses a DHCP-based cable network (DOCSIS) and so when your router connects it asks for an IP and is given it, from what I'm assuming is kit at the head end (not cabinet). This is where the problems start, they don't have the ability to add a static allocation from here (Probably how their pools of IPs are allocated to local exchanges/cabinets, and/or their DHCP servers).

Inside the street cabinet doesn't reveal much active equipment, the magnavox amplifier unit is line powered from the main co-ax uplinks (Big chunky cables coming in bottom left) and then split to the cable junctions to end users (bottom middle) and potentially legacy twin-core/pair copper to the right.



This setup of sharing the residential with business then causes them a headache when a business customer asks for a static IP, how to solve that with the DOCSIS implementation Virgin Media has used? What they do is create a GRE tunnel using the business hub (Hitron router) to their datacentre, where they allocate the public IP on that end of the tunnel and allow it to connect out from the datacentre. This also allows them to bypass any content control, filtering, etc, as it is then emerging from the tunnel at their datacentre rather than the regular pool.

Most of you are probably hearing the alarm bells ringing now. GRE tunnel to datacentre, so the tunnel is established using the Hitron router on your premises and breakout is somewhere in the VM datacentre network. This to me shows several potential problems, the first being MTU.
Over the GRE tunnel MTU can and will be reduced, my conversations with VM support suggest this is down to 1440 but I've not fully tested this.
The second is that I'm not not sure where/what is doing the NAT for our connection. Although the Hitron allows me to setup DMZ, port forwarding, etc, I'm not clear if this is working through the GRE tunnel or not! This also introduces a further complication, you CANNOT use the Hitron router in modem-only mode, so you HAVE to use the NAT functions on this router, again not good for a business class product aimed at people who would want to do their own NAT or control via their own server, etc. So you're stuck with the firewall and NAT functions on the Hitron, and whilst basic they seem to do what's needed.

That is until you start to use SIP/VOIP. This seems very problematic, as allocating the RTP data ports seems spotty, registration to a SIP gateway on udp/5060 also seems to be affected as sometimes it works, then stops and won't start again for a long period of time. This is regardless of if you setup your voip server as DMZ because some issues still remain.
Then you have the major showstopper I hit upon, after some arbitrary time the connection will drop (no surprise, they have to upgrade, have outages, etc) but when it comes back, SIP registrations will FAIL. For some reason packets don't make it out of the Virgin Media network. So from your originating server, you can tcpdump and see the traffic, but the receiving end doesn't see it. No matter what you do (reboot Hitron, reboot your server, re-recreate connections, etc) it won't recover, and this brings me to a theory. There is some sort of session being held on the remote end of that GRE tunnel for your static IP. And as such it is blocking/stopping new sessions to the same destination IP, causing your SIP registration to fail and your VOIP solution to stop working. My guess on this is because it depends on what is on the other end of that GRE tunnel, and what it's involvement is in your connection. It may be some type of firewall, in which case it's trying to keep state of UDP sessions and failing miserably. It may be a router, in which case I'd not be expecting the issues we have seen, but it's still possible.

So far Virgin Media have confirmed that there is a known issue with static IP addresses on their business cable solution, but before you buy they won't go into much detail, and after purchase unfortunately you're stuck in this solution where you can either live with the issues on static IP, or drop back to a dynamic DHCP allocated IP and not have the GRE tunnel.

I suspect the solution to this is to move to a dynamic IP on the service and then switch to modem-only mode so nothing is doing NAT on the connection. I'll post back on further diagnostics that I carry out to further explore what the issue is and if it can be worked around. So far no work-arounds I've tried have worked, other then connection out using an alternative UDP port for SIP (Which most SIP providers won't do).

--Update--
After some conversations with VM they have switched the connection to a dynamic IP. Beware, as when they do this they reconfigure things their side, tell you to reboot the cable modem and it takes you offline. That's because the GRE tunnel information is still coded into your cable modem. Factory resets using the front button, rear pin press button and control panel interface for factory reset doesn't seem to work (Doesn't appear to factory reset at all as settings do not revert to when shipped) so this causes you outage. In this case Virgin Media had to send an engineer out to do another reset to the cable modem to resolve this. When they did that the modem connected up and got an IP from the dynamic local pool.
After this connection was restored, and sure enough the VOIP sessions re-established and maintained their connection to the VOIP provider. (Again the VOIP server was setup as DMZ target on the Virgin Media Hitron hub) So this has appeared to solve the issue with SIP registrations over the service.


Thursday, 11 February 2016

Cheap Chinese IP cameras (Wanscam) and their network activity

Here's a curious item, I've had quite a few of these over the years, not just IP Cameras but home automation relays, thermostats, etc. They all generally have IP connectivity and I duly just connect them into my LAN and start to use them.
However, I've been looking at why my internet connection keeps spiking and trying to cut down on anything not required to chatter to the internet, as my home connection is a very poor ADSL that cannot sustain much traffic, so cutting any junk out will help lots!

So running packet traces on my public connection (All devices on the LAN default gateway to my home server (A linux server) and then this does NAT to the public IPs I have), and started to notice a pattern of requests constantly going out over the public interface.

The first one is shown below:

12:33:47.668240 IP OBSCURED.2614 > OBSCURED.domain: 6038+ A? www.db-power.com. (34)
12:33:47.686932 IP OBSCURED.domain > OBSCURED.2614: 6038 NXDomain* 0/0/0 (34)
12:33:47.690643 IP OBSCURED.2614 > OBSCURED.domain: 6039+ A? www.db-power.com. (34)
12:33:47.709829 IP OBSCURED.domain > OBSCURED.2614: 6039 NXDomain* 0/0/0 (34)
12:33:47.723202 IP OBSCURED.2614 > OBSCURED.domain: 6042+ A? www.db-power.com. (34)
12:33:47.742344 IP OBSCURED.domain > OBSCURED.2614: 6042 NXDomain* 0/0/0 (34)
12:33:47.746075 IP OBSCURED.2614 > OBSCURED.domain: 6043+ A? www.db-power.com. (34)
12:33:47.764766 IP OBSCURED.domain > OBSCURED.2614: 6043 NXDomain* 0/0/0 (34)
12:33:47.777307 IP OBSCURED.2614 > OBSCURED.domain: 6046+ A? www.db-power.com. (34)
12:33:47.796293 IP OBSCURED.domain > OBSCURED.2614: 6046 NXDomain* 0/0/0 (34)
12:33:47.801283 IP OBSCURED.2614 > OBSCURED.domain: 6047+ A? www.db-power.com. (34)
12:33:47.819959 IP OBSCURED.domain > OBSCURED.2614: 6047 NXDomain* 0/0/0 (34)
12:33:47.832231 IP OBSCURED.2614 > OBSCURED.domain: 6050+ A? www.db-power.com. (34)
12:33:47.850810 IP OBSCURED.domain > OBSCURED.2614: 6050 NXDomain* 0/0/0 (34)

(I've OBSCURED my IP/hostname and my upstream DNS providers resolver here)
So what was happening is something was asking for the dns entry for www.db-power.com and constantly being told this doesn't exist. So this capture was done on my external interface, switching to my internal interface (for my LAN on the NAT server) I then tracked this down to an IP Camera was making this constant request. Logging into the web interface of the camera I couldn't find anything referring to this, but clearly it has some process where it tries to 'phone home' so that was the first one tracked down. I simply put a block in my local caching DNS to stop this external request, so that stopped it from asking externally and wasting bandwidth (It was doing this every 60 seconds).

The next one I then spotted was this:


12:38:47.545925 IP OBSCURED.3203 > ec2-54-243-97-206.compute-1.amazonaws.com.32100: UDP, length 24
12:38:47.546216 IP OBSCURED.3203 > ec2-54-251-113-251.ap-southeast-1.compute.amazonaws.com.32100: UDP, length 24
12:38:47.546402 IP OBSCURED.3203 > ec2-54-246-107-165.eu-west-1.compute.amazonaws.com.32100: UDP, length 24
12:38:49.354353 IP OBSCURED.3208 > ec2-54-243-97-206.compute-1.amazonaws.com.32100: UDP, length 44
12:38:49.354574 IP OBSCURED.3208 > ec2-54-251-113-251.ap-southeast-1.compute.amazonaws.com.32100: UDP, length 44
12:38:49.354774 IP OBSCURED.3208 > ec2-54-246-107-165.eu-west-1.compute.amazonaws.com.32100: UDP, length 44
Again OBSCURED was my cameras IP address. So this was more interesting. The camera was constantly (every minute again) trying to contact something in the Amazon AWS cloud. This indicates somebody has a server on Amazon's cloud that these units are contacting. I have two IP cameras both different manufacturers and different interfaces but they were both doing this, so there must be a common firmware package being used that was making this communication.
Interestingly the machine at the other end was refusing the UDP connection, but still, why was it there?
First thing, I blocked this from making it out of my firewall, then set about looking to see what it's doing. The payload is UDP therefore it's stateless, so most likely it's just squirting a bit of info about itself (perhaps firmware, version, date/time) back to it's manufacturer. Possibly it was also used as a 'cloud' solution for viewing or managing your camera as I have seen this principle, but again this was all disabled on my unit, but this suggests even when disabled it still makes the initial phone-home even if nothing else was passed. As you can also see it's trying to talk to a few IP addresses in Amazon. I'm unsure if these are hard-coded in or if it's a DNS request the device makes, so next step was to sniff it's traffic and see what DNS requests it was making. In this case it didn't appear to be making any DNS requests, which suggests these hosts are hardcoded into it.
Viewing the actual payload didn't help much either:


13:17:00.226305 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 52)
    OBSCURED.3203 > 54.246.107.165.32100: [udp sum ok] UDP, length 24
 0x0000:  4500 0034 0000 4000 4011 a06c c0a8 3709  E..4..@[email protected].
 0x0010:  36f6 6ba5 0c83 7d64 0020 f89b f191 0014  6.k...}d........
 0x0020:  4a57 4556 0000 0000 0003 95fe 4545 4343  JWEV........EECC
 0x0030:  4300 0000                                C...
13:17:01.942156 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    OBSCURED.3208 > 54.243.97.206.32100: [udp sum ok] UDP, length 44
 0x0000:  4500 0048 0000 4000 4011 aa32 c0a8 3709  E..H..@[email protected].
 0x0010:  36f3 61ce 0c88 7d64 0034 bf8e f110 0028  6.a...}d.4.....(
 0x0020:  4a57 4556 0000 0000 0003 95fe 4545 4343  JWEV........EECC
 0x0030:  4300 0000 0200 0721 0002 880c 0937 a8c0  C......!.....7..
 0x0040:  0000 0000 0000 0000                      ........
13:17:01.942371 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto UDP (17), length 72)
    OBSCURED.3208 > 54.251.113.251.32100: [udp sum ok] UDP, length 44
 0x0000:  4500 0048 0000 4000 4011 99fd c0a8 3709  E..H..@[email protected].
 0x0010:  36fb 71fb 0c88 7d64 0034 af59 f110 0028  6.q...}d.4.Y...(
 0x0020:  4a57 4556 0000 0000 0003 95fe 4545 4343  JWEV........EECC
 0x0030:  4300 0000 0200 0721 0002 880c 0937 a8c0  C......!.....7..
 0x0040:  0000 0000 0000 0000
Which I couldn't work out what this was containing. It was always the same length and contained similar information, only one or two characters changed, but I couldn't see any correlation between what the camera was doing and this value, so I don't think I'll spot what it is.

One final thing, was I port scanned the camera, which showed 23 (telnet), 99 (metagram), 8600 (asterisk). Now the first telnet port was correct as I could telnet and get a login prompt. Port 99 was the web port (I'd changed it to that) and 8600 seemed to answer telnet then immediately close it again. So I thought I'd try telnet:


Escape character is '^]'.

(none) login: root
Password: 


BusyBox v1.12.1 (2012-11-20 15:16:24 CST) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# ps
  PID USER       VSZ STAT COMMAND
    1 root      1528 S    init       
    2 root         0 SWN  [ksoftirqd/0]
    3 root         0 SW<  [events/0]
    4 root         0 SW<  [khelper]
    5 root         0 SW<  [kthread]
    6 root         0 SW<  [kblockd/0]
    7 root         0 SW<  [khubd]
    8 root         0 SW<  [kswapd0]
    9 root         0 SW   [pdflush]
   10 root         0 SW   [pdflush]
   11 root         0 SW<  [aio/0]
   12 root         0 SW<  [scsi_tgtd/0]
   13 root         0 SW   [mtdblockd]
   14 root         0 SW<  [kmmcd]
   20 root      1092 S    nvram_daemon 
   23 root         0 SWN  [jffs2_gcd_mtd6]
   25 root         0 SWN  [jffs2_gcd_mtd7]
   28 root      1528 R    telnetd 
   29 root      1696 S    /system/system/bin/daemon.v5.12 
   30 root      1480 S    /system/system/bin/cmd_thread 
   31 root      1480 S    /system/system/bin/gmail_thread 
   32 root      1532 S    /bin/sh 
   33 root      1696 S    /system/system/bin/daemon.v5.12 
   36 root      1480 S    /system/system/bin/cmd_thread 
   39 root      1480 S    /system/system/bin/gmail_thread 
   40 root      1696 S    /system/system/bin/daemon.v5.12 
   41 root      1696 S    /system/system/bin/daemon.v5.12 
   43 root      1480 S    /system/system/bin/cmd_thread 
   44 root      1480 S    /system/system/bin/gmail_thread 
   50 root         0 SW   [RtmpCmdQTask]
   51 root         0 SW   [RtmpWscTask]
   89 root      1696 S    /system/system/bin/daemon.v5.12 
  135 root      1524 S    /sbin/udhcpc -i eth2 -n 
  140 root      1696 S    /system/system/bin/daemon.v5.12 
  141 root      1696 S    /system/system/bin/daemon.v5.12 
  142 root      1696 S    /system/system/bin/daemon.v5.12 
  145 root      1696 S    /system/system/bin/daemon.v5.12 
  146 root      1696 S    /system/system/bin/daemon.v5.12 
  147 root      1696 S    /system/system/bin/daemon.v5.12 
  151 root     11528 S    /system/system/bin/encoder 
  158 root     11528 S    /system/system/bin/encoder 
  159 root     11528 S    /system/system/bin/encoder 
  160 root     11528 S    /system/system/bin/encoder 
  161 root     11528 S    /system/system/bin/encoder 
  164 root     11528 S    /system/system/bin/encoder 
  165 root     11528 S    /system/system/bin/encoder 
  167 root     11528 S    /system/system/bin/encoder 
  168 root     11528 S    /system/system/bin/encoder 
  169 root     11528 S    /system/system/bin/encoder 
  170 root     11528 S    /system/system/bin/encoder 
  171 root     11528 S    /system/system/bin/encoder 
  172 root     11528 S    /system/system/bin/encoder 
  173 root     11528 S    /system/system/bin/encoder 
  178 root     11528 S    /system/system/bin/encoder 
  179 root     11528 S    /system/system/bin/encoder 
  180 root     11528 S    /system/system/bin/encoder 
  181 root     11528 S    /system/system/bin/encoder 
  182 root     11528 S    /system/system/bin/encoder 
  183 root     11528 S    /system/system/bin/encoder 
  184 root     11528 S    /system/system/bin/encoder 
  185 root     11528 S    /system/system/bin/encoder 
  186 root     11528 S    /system/system/bin/encoder 
  187 root     11528 S    /system/system/bin/encoder 
  188 root     11528 S    /system/system/bin/encoder 
  189 root     11528 S    /system/system/bin/encoder 
  190 root     11528 S    /system/system/bin/encoder 
  191 root     11528 S    /system/system/bin/encoder 
  192 root     11528 S    /system/system/bin/encoder 
  193 root     11528 S    /system/system/bin/encoder 
  194 root     11528 S    /system/system/bin/encoder 
  195 root     11528 S    /system/system/bin/encoder 
  196 root     11528 S    /system/system/bin/encoder 
  197 root     11528 S    /system/system/bin/encoder 
  198 root     11528 S    /system/system/bin/encoder 
  199 root     11528 S    /system/system/bin/encoder 
  200 root     11528 S    /system/system/bin/encoder 
  201 root     11528 S    /system/system/bin/encoder 
  202 root     11528 S    /system/system/bin/encoder 
  203 root     11528 S    /system/system/bin/encoder 
  204 root     11528 S    /system/system/bin/encoder 
  205 root     11528 S    /system/system/bin/encoder 
  206 root     11528 S    /system/system/bin/encoder 
  207 root     11528 S    /system/system/bin/encoder 
  208 root     11528 S    /system/system/bin/encoder 
  209 root     11528 S    /system/system/bin/encoder 
  210 root     11528 S    /system/system/bin/encoder 
  211 root     11528 S    /system/system/bin/encoder 
  213 root     11528 S    /system/system/bin/encoder 
  214 root     11528 S    /system/system/bin/encoder 
  215 root     11528 S    /system/system/bin/encoder 
  216 root     11528 S    /system/system/bin/encoder 
  217 root     11528 S    /system/system/bin/encoder 
  218 root     11528 S    /system/system/bin/encoder 
  221 root     11528 S    /system/system/bin/encoder 
  222 root     11528 S    /system/system/bin/encoder 
  223 root     11528 S    /system/system/bin/encoder 
  224 root     11528 S    /system/system/bin/encoder 
  225 root     11528 S    /system/system/bin/encoder 
  226 root     11528 S    /system/system/bin/encoder 
  227 root     11528 S    /system/system/bin/encoder 
  228 root     11528 S    /system/system/bin/encoder 
  229 root     11528 S    /system/system/bin/encoder 
  230 root     11528 S    /system/system/bin/encoder 
  231 root     11528 S    /system/system/bin/encoder 
  236 root     11528 S    /system/system/bin/encoder 
  237 root     11528 S    /system/system/bin/encoder 
  238 root     11528 S    /system/system/bin/encoder 
  239 root     11528 S    /system/system/bin/encoder 
  240 root     11528 S    /system/system/bin/encoder 
  252 root     11528 S    /system/system/bin/encoder 
  253 root     11528 S    /system/system/bin/encoder 
 1891 root     11528 S    /system/system/bin/encoder 
 1892 root     11528 S    /system/system/bin/encoder 
 3553 root     11528 S    /system/system/bin/encoder 
 3554 root     11528 S    /system/system/bin/encoder 
 3821 root      1532 S    -sh 
 3824 root      1528 R    ps 

As you can see, I logged in! I tried using the username and password I'd set using the web interface, but this failed. So I tried the default it shipped with, 'root' and password '123456' and it let me in! So it seems these all have this as a default, and once in it runs a basic busybox linux system. I've not had much time to look any further, but if you're interested here is the filesystem and what looks like the init script that is ran at system bootup to start the daemons
# df
Filesystem           1k-blocks      Used Available Use% Mounted on
rootfs                    3008      3008         0 100% /
/dev/root                 3008      3008         0 100% /
/dev/mtdblock6            3072      1892      1180  62% /system
/dev/mtdblock7             512       208       304  41% /param
# ls /system
system    daemon    Wireless  init      www
# cat /system/init/ipcam.sh 
export LD_LIBRARY_PATH=/system/system/lib:$LD_LIBRARY_PATH
export PATH=/system/system/bin:$PATH
telnetd
/system/system/bin/daemon.v5.12 &
/system/system/bin/cmd_thread &
/system/system/bin/gmail_thread &
# 
So there looks to be 3 daemons that I'd like to have a look inside, so may do at some point in the future.