Effectively Hiding Behind Cloudflare

I am an avid user of Cloudflare's free service. They offer DNS name serving, caching, and some basic request rewriting and security features for $0. Since Cloudflare's features rely on all HTTP/S traffic hitting their own servers before being proxied on to yours, it is particularly important - especially from a security standpoint - that the real IP address of your server remains hidden to bad actors. This isn't as simple as it may seem at first.

The Basics

Getting the basics in place, of course, is not hard, and there are various guides across the web on getting these right. The first thing is to ensure you have no sub-domains pointing directly at your server IP address without Cloudflare protection. This is done within the Cloudflare interface itself on the DNS settings page.

The following thing to do is to remove any publicly accessible files which give away the server address. You will need to be familiar with your application here, but any page containing the well known phpinfo() function is an obvious example for PHP-based websites. This should of course be removed or password protected regardless of whether you're behind Cloudflare or not.

Next up is to ensure any outgoing communications from your server's application(s) are secured. This means all emails are sent via SMTP servers which do not reveal the original sender's IP in the message headers. It also means disabling services such as PingBacks, and accessing only trusted third party APIs (via cURL, for example) that aren't going to reveal your origin server to any other users.

The last step is usually then changing your server's IP address, so that any historical records of your server before moving on to Cloudflare's service are then rendered useless to an attacker who may dig them up. However, there are actually some additional steps you will need to take before implementing this.

Admin Tools and Dashboards

You may be running a web-based admin tool of some kind. Examples might be a local web analyzer which contains referral links to your website, or a webmail inbox. Since it's a backend feature just for you and not your users, you might be tempted to access it directly by the server's IP address and not via Cloudflare. Think again. By following any links that are in emails or web dashboards, you potentially send your server's IP address (via the HTTP referer header) back to someone else's system for them to analyse. You should always access all web-based content on your server via Cloudflare to maintain security.

IP Scanners

The device search engine Shodan has been around for a few years now, but has only more recently gained notoriety for helping to uncover massive vulnerabilities across servers and other devices sitting on the IPv4 address space. Indeed, Shodan regularly scans the entire address space across various ports, indexing certain features of the services it discovers so that people can search on them. Your web server is one such service, sitting amongst a fairly digestible address space of about 4.3 billion public IPs in total.

Searching Shodan for the term "cloudflare" produces a lot of results. Plenty of these results are from CloudFlare's own address space, but plenty are not. By drilling down to only HTTP/S results hosted with a known hosting provider such as Digital Ocean, we can find some web servers which are set to redirect all incoming HTTP (port 80) requests to their Cloudflare protected domain name on HTTPS. This provides an instant link between the domain name and the origin server.

For an attacker wanting to specifically target a particular domain name, Shodan indexes other fields in device responses. One is able to search on domain names and get results due to cookies being inexplicably set in the response for all visitors, redirects as above, or rather oddly, occasional redirect results instead of the expected redirect statement itself. Perhaps Shodan still has some indexing bugs.

One thing Shodan does not do is allow you to search directly on the certificate domain name. This means that while plenty of Cloudflare users may have their own SSL certificate or a Cloudflare origin certificate with their domain name mentioned on it, Shodan will log it, but an attacker can't use Shodan to search for it directly to discover the server IPs using that certificate. This also goes for actual website content: Shodan scans and logs website content to a degree, but it is not searchable.

This is where Censys comes in. Like Shodan, Censys scans the entire IPv4 address space across the internet, but unlike Shodan, it indexes certificate domain names and content. This means that searching for any domain name on Censys can bring up any server IP which is serving a certificate using that domain name. This will include origin servers using Cloudflare origin certificates, or of course their own certificates if they've decided to use them, say from Let's Encrypt. Of course an alternative solution here would be to not use a certificate at all and use CloudFlare's Flexible SSL option, but this is known to be insecure and actively exploited.

Censys also indexes the entire HTTP response of its scan, including the body - the HTML content on whatever web server it finds. This means an attacker can simply search for any IP address that is serving content which so much as mentions a given phrase.

How Not to Deal with IP Scanners

You might be thinking that this could be solved by the web server. Apache and NGINX can be configured to ensure incoming connections without a domain name (such as an IP scanning bot) will just default to a particular vhost or server block which serves a blank page or an error. This would mean the only requests which are properly served are those which include your domain name (and hence, are coming via Cloudflare).

For regular HTTP connections on port 80 this seems workable, but throwing HTTPS in to the mix brings issues. Because the TLS negotiation happens before the HTTP connection is established, some sort of certificate must be presented. This certificate must not have your domain name on it, or the game is up. In fact, it shouldn't be for any domain at all, so a self signed certificate attached to the '*' wildcard is the way to go here.

Even if you are OK with such a configuration mess, this method also allows for guesswork. Should an attacker believe they have some idea of your server IP, they can validate it by sending an HTTP request to the IP address with your domain name set in the Host header. Your server will then duly serve up your website in response. An automated method of sending out such a request to many IPv4 addresses until a desired response is received is certainly not out of the question.

Whitelisting Cloudflare in iptables

The solution, then, is to block such requests from the server entirely. Naturally, the safest way to do this is to block all incoming requests to your server that are not from Cloudflare. We don't want to do this on all ports, of course (SSH isn't served through Cloudflare anyway), so for a typical web server which only serves content on port 443, we can whitelist Cloudflare's IP ranges accordingly.

This server uses something very similar to the following iptables rules:

sudo iptables -I INPUT 1 -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

sudo iptables -A INPUT -p tcp -s 103.21.244.0/22 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 103.22.200.0/22 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 103.31.4.0/22 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 104.16.0.0/12 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 108.162.192.0/18 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 131.0.72.0/22 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 141.101.64.0/18 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 162.158.0.0/15 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 172.64.0.0/13 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 173.245.48.0/20 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 188.114.96.0/20 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 190.93.240.0/20 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 197.234.240.0/22 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 198.41.128.0/17 --dport 443 -j ACCEPT
sudo iptables -A INPUT -p tcp -s 199.27.128.0/21 --dport 443 -j ACCEPT

sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -j REJECT

The above rules allow existing connections, then whitelist new HTTPS connections from Cloudflare's IP ranges, before allowing SSH connections and rejecting everything else. Note that port 80 isn't mentioned as it is not being served at all (I am using Cloudflare's strict SSL option). To redirect incoming HTTP requests to HTTPS, Cloudflare's page rules should instead be used via their web interface.

These same iptables rules could also be applied again using ip6tables and Cloudflare's IPv6 ranges if desired. At the moment however, I am not aware of any services that are attempting the impossible task of scanning the entire IPv6 range of 340 undecillion addresses.

Note that Cloudflare actively discouraged the above for free tier plan users, as it would mean a site would go completely offline in an attack if they were kicked off Cloudflare. After I contacted Cloudflare last month to point out the discrepancy between that policy and the issues caused by IPv4 scanners, they appear to have updated their response.

Only after you have protected your server from IP scanners should you commit to the final step of changing your server's public IPv4 address.