IP (Internet Protocol) Address is a unique identity number assigned to every device over the Internet connected to the computer network that uses the Internet protocol for communication.
Every bot that visits your website comes through an IP Address.
All the external referring backlinks have a referring IP. IP Addresses are assigned to the list of domains.
The below screenshot shows how I found backlinks emerging from referring IPs, this referring IPs list can be downloaded from Semrush for your domain.
Steps to replicate:
- Semrush Backlink Analytics
- Scan your domain in Backlink Analytics
- Click on the Referring IPs section to get this list
Visits to the website, whether from users or bots, can help or hurt a website; the same goes for the referring IPs.
Here are some common instances of how it hurts
- Someone using a spam bot conducts a DDOS attack on your website to crash your server.
- Spammy external sites belonging to a referring IP worsen your backlink profile.
Now I used to conclude that Google largely ignores spammy backlinks and we don’t need to do anything about it but then this screenshot by Marie Haynes changed my perspective.
Source: (MarieHaynes)
Now let’s jump to the part that you have been waiting for. I have built a Python Script that detects IP addresses with higher abuse scores. For the detected IPs you can do these two things:
- Block those IP hits from the CDN Firewall so they don’t crash your server
- Disavow websites belonging to those IPs (after a thorough investigation, but them belonging to an abused IP list is definitely a check & a red flag)
How to download IPs from Cloudflare?
To download the IP & other details from Cloudflare for your website, the steps are as follows:
Go to Account > Analytics & Logs > Account Analytics In the top right section of the screenshot below, you can see a button ‘Download Data: All sites for account’ You have to click that to download the data & that data will contain the IP data.
How to block IPs from Cloudflare?
To block IPs in Cloudflare:
Go to Security > WAF > Create a rule
This screenshot shows how to create the rule & block the IP from hitting your website.
I have used Google Colab IDE for this script.
Step 1: Import function to upload the txt file containing the list of IP addresses
from google.colab import files
uploaded = files.upload()
Step 2: Leverage abuseipdb.com API & use python code to make API requests
TL;DR;
AbuseIPDB is a project dedicated to identifying & combatting the hackers & spam attacks that the internet faces. People are actively reporting abused IPs regularly.
You can view the statistics on this website to see the reports by hours, months, and year. It even visualizes the country from where the attacks are emerging.
import requests
import json
def check_abuse_score(api_key, file_path):
# Defining the api-endpoint
url = 'https://api.abuseipdb.com/api/v2/check'
headers = {
'Accept': 'application/json',
'Key': api_key
}
# Reading IP addresses from the file
with open(file_path, 'r') as file:
ip_addresses = file.read().splitlines()
# Collecting IP addresses with abuse score greater than 89
high_confidence_ips = []
for ip in ip_addresses:
querystring = {'ipAddress': ip}
response = requests.request(method='GET', url=url, headers=headers, params=querystring)
decoded_response = json.loads(response.text)
# Check if 'data' key is present in the response
if 'data' in decoded_response and 'abuseConfidenceScore' in decoded_response['data']:
abuse_confidence_score = decoded_response['data']['abuseConfidenceScore']
if abuse_confidence_score > 89:
high_confidence_ips.append({'ipAddress': ip, 'abuseConfidenceScore': abuse_confidence_score})
else:
print(f"Error processing IP address {ip}: {decoded_response}")
# Printing the list of IP addresses with abuse score greater than 89
print("IP Addresses with Abuse Confidence Score > 89:")
for entry in high_confidence_ips:
print(f"IP Address: {entry['ipAddress']}, Confidence Score: {entry['abuseConfidenceScore']}")
# Replace 'YOUR_OWN_API_KEY' with your actual API key
api_key = 'Add_your_API'
# Replace 'path/to/your/file.txt' with the actual path to your file containing IP addresses
file_path = '/content/List_Ips.txt'
# Calling the function
check_abuse_score(api_key, file_path)
Note: abuseipdb.com provides 1000 credits on a daily basis.
Step 3: See the abused IP’s printed on your IDE
In the above screenshot, we can see how the IP addresses with higher abuse rates have been highlighted.
The API can be trusted because people from all over the world are reporting abusive IPs every time. Please refer to the screenshot below.
Let’s say you witnessed your server getting crashed on certain days but it isn’t correlating with the visits figure from GA4.
This is when you can download the IPs from logs & utilize this Python script to detect IPs with abusive scores if any of those are causing a load on your server.
