Recently I was really annoyed by my ISP (TalkTalk @ UK).
In short: They are intercepting traffic and doing deep packet inspection without any warning or approval.
But wait, there’s more: In general they monitor web traffic (read: the data) and after intercepting an HTTP request the replay that (yes.. they replay the request).
Here’s an example:
78.149.130.80 - - [12/Mar/2012:22:47:23 +0100] "GET /korokokokokoLALALALA HTTP/1.1" 404 536 "-" "Wget/1.13.4 (linux-gnu)" 62.24.252.133 - - [12/Mar/2012:22:47:55 +0100] "GET /korokokokokoLALALALA HTTP/1.0" 404 498 "http://<removed>/korokokokokoLALALALA" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)"
The first request was performed by me and it was followed by a second one 32 seconds later (for the unbeliever: this is 100% reproducible). The IP address of the offender is always the same (62.24.252.133).
Digging a bit on this I found these:
- http://talktalkmembers.net/forums/showthread.php?t=54112
- http://www.whatdotheyknow.com/request/the_last_3_months_of_tt_ico
I don’t think there are enough things I can say about this. However, here are a couple of them:
- They shouldn’t do that
- They are a bunch of $#&@%)&#$%
- This actually doubles the web traffic that originates from the talktalk network
- It can be really harmful as it is possible to trigger a delete action (i.e. when GET is used instead of POST)
- It is completely unethical
- It is performed even when you have all the security features disabled
- Did I say that they are a bunch of @$#)*&%*#$ ?
In any case, I started exploiting this a bit:
Fortunately I’ve a domain under my control and a couple of servers. As such I used one of them that is already running apache and added this rule:
iptables -I OUTPUT -d 62.24.252.133 -p tcp --sport 80 -m tcp --tcp-flags SYN '' -j REJECT --reject-with tcp-reset
After that I ran this from a PC at home:
for b in $(seq 0 9) ; do ( ( for i in $(seq 22${b}00 22${b}99) ; do wget http://xxx.xxx.xxx/bad-talktalk-bad-bad-$i ; done ) & ) ; done
The idea is to create an iptables rule at the server that matches outgoing TCP segments without the SYN flag (iow: segments after the initial 2 SYNs of the handshake) and reset that connection. The result is that when that host (62.24.252.133) tries to re-fetch the page (i.e. replay the request):
- It initiates the TCP connection sending a SYN to the server which is accepted
- The server replies with SYN+ACK which is normally sent
- The offender receives the SYN+ACK so it goes to the ESTABLISHED state, sending an ACK plus some data
- The server receives the data and responds with an ACK
- The iptables rule takes effect, dropping the packet and responding to the server (itself) with a reset
After the above steps, the server closes the connection abnormally and the offender stays with a fully open connection and keeps trying to send the data.
The rational behind that was to exhaust the offender’s TCP ports by creating >60000 connections.
The good news is that the theory behind that works. The bad news is that the offender is very slow (most probably on purpose). The good news after that is that it keeps a backlog of the connections and tries to perform them all (or a big part of them).
My tests showed the offender trying to perform connections up to an hour after that.
Here are the results of itpables’ accounting:
Chain OUTPUT (policy ACCEPT 75792 packets, 16M bytes) pkts bytes target prot opt in out source destination 14600 584K REJECT tcp -- * * 0.0.0.0/0 62.24.252.133 Chain INPUT (policy ACCEPT 0 packets, 0 bytes) pkts bytes target prot opt in out source destination 28462 6088K ACCEPT tcp -- * * 62.24.252.133 0.0.0.0/0
The first one is the output rule. The second one is a rule I added for accounting purposes.
So we ended up having 14600 resets sent and 28400 packets from the offender. (note: I only created about 3000 connections)
The day is over but I have other plans as well:
- Create a bunch (i.e. 100) alternate DNS names for the server and perform the requests against them, pushing the offender to perform more requests in parallel
- Write a python program that operates at client side and server side constructing fake TCP packets with predefined sequence numbers and fake IP addresses so that the offender will believe that it will have to follow more than one users.
- Study whether the offender it intelligent. It is possible that it only inspects packet payloads for HTTP requests instead of fully reconstructing the whole TCP stream. In that case I’ll only have to send a large number of data packets with HTTP requests
62.24.252.133 – – [12/Mar/2012:22:47:55 +0100] “GET /korokokokokoLALALALA HTTP/1.0” 404 498 “http://srv3.v13.gr/korokokokokoLALALALA” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; InfoPath.2)”
What you did is totally awesome!
LikeLike
Found your blog post after discovering this replay on a brand new Azure VM instance. I see 80.40.134.120 as the IP of the TalkTalk replay server.
LikeLike
TalkTalk get so much bad press, but they really do offer one of the most comprehensive services compared to sky or virgin, at a fraction of the price! This just confirms that even more!
LikeLike
No they don’t. You just proved it.
You (talk-talk) didn’t even bother understanding what’s being said here. That’s why people are leaving you. Because one has to deal with this kind of stuff when a technical issue arises.
LikeLike
They’re still up to this shenanigans, and from the same IP as well.
I’m curious — do they follow HTTP redirects? Can we find an HTTP redirect on their own site that we can get their own bot to visit? It’d be hilarious if they could DoS themselves.
Also, even worse is that they request robots.txt now and then immediately ignore it — /that/ is just adding insult to injury.
LikeLike