Saturday, October 5, 2019

Defending against attack in AWS

I operate a LAMP stack in AWS. Full details for the setup are here: PHP MySQL (MariaDB) in AWS Micro with Amazon Linux 2.

Recently I've been under persistent attack. I'll post details here in case it's helpful.

First you have to notice that you're under attack. In my case the high network out alarm was triggered.

[right-click] EC2 Instance > CloudWatch Monitoring > Add/Edit Alarms > Create Alarm

[checked] send a notification to: your contact info
Whenever: Average of Network Out
Is: >= 150000 Bytes
For at least 1 consecutive period of 6 hours

Create Alarm > Close

This led me to look at my apache logs. There are lots of entries like these:

148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=301111111111111%22%20UNION%20SELECT%20CHAR(45,120,49,45,81,45),CHAR(45,120,50,45,81,45),CHAR(45,120,51,45,81,45),CHAR(45,120,52,45,81,45),CHAR(45,120,53,45,81,45),CHAR(45,120,54,45,81,45),CHAR(45,120,55,45,81,45),CHAR(45,120,56,45,81,45),CHAR(45,120,57,45,81,45),CHAR(45,120,49,48,45,81,45),CHAR(45,120,49,49,45,81,45),CHAR(45,120,49,50,45,81,45),CHAR(45,120,49,51,45,81,45),CHAR(45,120,49,52,45,81,45),CHAR(45,120,49,53,45,81,45),CHAR(45,120,49,54,45,81,45),CHAR(45,120,49,55,45,81,45),CHAR(45,120,49,56,45,81,45),CHAR(45,120,49,57,45,81,45),CHAR(45,120,50,48,45,81,45),CHAR(45,120,50,49,45,81,45),CHAR(45,120,50,50,45,81,45),CHAR(45,120,50,51,45,81,45),CHAR(45,120,50,52,45,81,45),CHAR(45,120,50,53,45,81,45),CHAR(45,120,50,54,45,81,45),CHAR(45,120,50,55,45,81,45),CHAR(45,120,50,56,45,81,45),CHAR(45,120,50,57,45,81,45),CHAR(45,120,51,48,45,81,45),CHAR(45,120,51,49,45,81,45),CHAR(45,120,51,50,45,81,45),CHAR(45,120,51,51,45,81,45),CHAR(45,120,51,52,45,81,45),CHAR(45,120,51,53,45,81,45),CHAR(45,120,51,54,45,81,45),CHAR(45,120,51,55,45,81,45),CHAR(45,120,51,56,45,81,45),CHAR(45,120,51,57,45,81,45),CHAR(45,120,52,48,45,81,45),CHAR(45,120,52,49,45,81,45),CHAR(45,120,52,50,45,81,45),CHAR(45,120,52,51,45,81,45),CHAR(45,120,52,52,45,81,45),CHAR(45,120,52,53,45,81,45),CHAR(45,120,52,54,45,81,45),CHAR(45,120,52,55,45,81,45),CHAR(45,120,52,56,45,81,45),CHAR(45,120,52,57,45,81,45),CHAR(45,120,53,48,45,81,45)%20--%20/*%20order%20by%20%22as HTTP/1.1" 301 1695 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=301111111111111%22%20UNION%20SELECT%20CHAR(45,120,49,45,81,45),CHAR(45,120,50,45,81,45),CHAR(45,120,51,45,81,45),CHAR(45,120,52,45,81,45),CHAR(45,120,53,45,81,45),CHAR(45,120,54,45,81,45),CHAR(45,120,55,45,81,45),CHAR(45,120,56,45,81,45),CHAR(45,120,57,45,81,45),CHAR(45,120,49,48,45,81,45),CHAR(45,120,49,49,45,81,45),CHAR(45,120,49,50,45,81,45),CHAR(45,120,49,51,45,81,45),CHAR(45,120,49,52,45,81,45),CHAR(45,120,49,53,45,81,45),CHAR(45,120,49,54,45,81,45),CHAR(45,120,49,55,45,81,45),CHAR(45,120,49,56,45,81,45),CHAR(45,120,49,57,45,81,45),CHAR(45,120,50,48,45,81,45),CHAR(45,120,50,49,45,81,45),CHAR(45,120,50,50,45,81,45),CHAR(45,120,50,51,45,81,45),CHAR(45,120,50,52,45,81,45),CHAR(45,120,50,53,45,81,45),CHAR(45,120,50,54,45,81,45),CHAR(45,120,50,55,45,81,45),CHAR(45,120,50,56,45,81,45),CHAR(45,120,50,57,45,81,45),CHAR(45,120,51,48,45,81,45),CHAR(45,120,51,49,45,81,45),CHAR(45,120,51,50,45,81,45),CHAR(45,120,51,51,45,81,45),CHAR(45,120,51,52,45,81,45),CHAR(45,120,51,53,45,81,45),CHAR(45,120,51,54,45,81,45),CHAR(45,120,51,55,45,81,45),CHAR(45,120,51,56,45,81,45),CHAR(45,120,51,57,45,81,45),CHAR(45,120,52,48,45,81,45),CHAR(45,120,52,49,45,81,45),CHAR(45,120,52,50,45,81,45),CHAR(45,120,52,51,45,81,45),CHAR(45,120,52,52,45,81,45),CHAR(45,120,52,53,45,81,45),CHAR(45,120,52,54,45,81,45),CHAR(45,120,52,55,45,81,45),CHAR(45,120,52,56,45,81,45),CHAR(45,120,52,57,45,81,45),CHAR(45,120,53,48,45,81,45)%20--%20/*%20order%20by%20%22as HTTP/2.0" 200 161040 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30 HTTP/1.0" 301 256 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%20AnD%20sLeep%283%29%20ANd%20%270%27%3D%270 HTTP/1.0" 301 303 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%26%26SlEEp%283%29 HTTP/1.0" 301 274 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%20AnD%20sLeep%283%29%20ANd%20%271 HTTP/1.0" 301 293 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%26%26sLEEp%283%29%26%26%271 HTTP/1.0" 301 287 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%00%27%7C%7CSLeeP%283%29%26%26%271 HTTP/1.0" 301 290 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%20AnD%20BeNChMaRK%282999999%2CMD5%28NOW%28%29%29%29 HTTP/1.0" 301 308 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%26%26BeNChMaRK%282999999%2CMD5%28NOW%28%29%29%29 HTTP/1.0" 301 305 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%20aND%20BeNChMaRK%282999999%2CMd5%28NoW%28%29%29%29%20AnD%20%271 HTTP/1.0" 301 324 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%26%26BeNChMaRK%282999999%2CmD5%28NOW%28%29%29%29%26%26%271 HTTP/1.0" 301 318 "-" "Opera/9.27"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20and%201%3D1 HTTP/1.1" 301 425 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20and%201%3D1 HTTP/2.0" 200 159770 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:04 +0000] "GET /michael/blog/post.php?y=30%27%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20%27x%27=%27x HTTP/1.1" 301 429 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:05 +0000] "GET /michael/blog/post.php?y=30%27%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20%27x%27=%27x HTTP/2.0" 200 159774 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:05 +0000] "GET /michael/blog/post.php?y=30%22%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20%22x%22=%22x HTTP/1.1" 301 429 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:50:05 +0000] "GET /michael/blog/post.php?y=30%22%20or%20(1,2)=(select*from(select%20name_const(CHAR(111,108,111,108,111,115,104,101,114),1),name_const(CHAR(111,108,111,108,111,115,104,101,114),1))a)%20--%20%22x%22=%22x HTTP/2.0" 200 159774 "-" "-"
148.72.104.162 - - [05/Oct/2019:13:51:56 +0000] "-" 408 - "-" "-"

First lets notice that HTTP/1.0 receives 301 "Moved Permanently".

"GET /michael/blog/post.php?y=30 HTTP/1.0" 301 256 "-" "Opera/9.27"

That's good, because we don't want to support it, but I'm a bit surprised, I thought the config would result in 403 "Forbidden"

RewriteCond %{SERVER_PROTOCOL} ^HTTP/1\.0$
RewriteRule ^ - [F]

Next let's see what the attacker is trying to achieve. It's obviously some kind of sql injection attack. CHAR(45,120,49,45,81,45) means "-x1-Q-".

Stack Overflow says this is just a test for injection. If the attacker can see the input in the served page then they know that injection is possible.

Fine. Your scripts are all correctly written and aren't prone to this kind of attack. But just ignoring the attack doesn't seem right.

What can we do about this? It might make you feel better to report the IP. Sometimes you'll see that others have reported it for the same behaviour.

Much better is to follow Jacob's Network ACLs guide and ban the IP. The steps in short are given below.

VPC > Network ACLs > Edit inbound rules > Add > 50, All, IP, Deny

But I've been attacked recently as follows:

104.160.27.35   - Sep 23, 2019
77.93.211.211   - Sep 25, 2019
148.66.157.162  - Sep 30, 2019
200.58.121.70   - Oct  2, 2019
209.236.118.251 - Oct  3, 2019
148.72.104.162  - Oct  5, 2019

Of course there are sophisticated solutions out there for this kind of thing, but I just have a hobby website.

You might know that certain strings will never appear appear in your URI. You could just forbid them.

  $lowUri = mb_strtolower($_SERVER['REQUEST_URI']);
  if ( your logic here )
  { 
    $attackIp = $_SERVER['REMOTE_ADDR'];
    http_response_code(403);
    exit;
  }

You could block the IP via apache, but to do that automatically would mean your PHP would need write access to server config which is a security risk.

My current scheme sends me a notification if an IP has launched a new attack within the last hour, and that IP gets 403 for the next hour. To further frustrate the bot you can put in a sleep before delivering the 403.

And for a bit of post analysis, here's how to see all entries from an attack IP:

sudo grep 77.93.211.211 /var/log/httpd/access_log

And here's how to see all the entries that don't contain a term.

This shows attacks not at the supplied path.

sudo grep 77.93.211.211 /var/log/httpd/access_log | grep -v "GET /michael/"

This shows attacks that weren't rejected by 403 or 301.

sudo grep 77.93.211.211 /var/log/httpd/access_log | grep -v " 403 " | grep -v " 301 "

aws
{ "loggedin": false, "owner": false, "avatar": "", "render": "nothing", "trackingID": "UA-36983794-1", "description": "", "page": { "blogIds": [ 697 ] }, "domain": "holtstrom.com", "base": "\/michael", "url": "https:\/\/holtstrom.com\/michael\/", "frameworkFiles": "https:\/\/holtstrom.com\/michael\/_framework\/_files.4\/", "commonFiles": "https:\/\/holtstrom.com\/michael\/_common\/_files.3\/", "mediaFiles": "https:\/\/holtstrom.com\/michael\/media\/_files.3\/", "tmdbUrl": "http:\/\/www.themoviedb.org\/", "tmdbPoster": "http:\/\/image.tmdb.org\/t\/p\/w342" }