1-Click Global Honeypot Network

    

By: Jason Alvarez,
Follow me on twitter @0xBanana


Deploying a global honeypot network in the cloud using terraform! Click it & ship it!

Hey everyone, in this post we’re going to expand on a few of my past posts. The goal today is to design and deploy a global honeypot network utilizing T-Pot and deployed using Terraform.

What is a honeypot?


This is something I wrote about in part 1 of this series, but if you missed it, lets go over the definition


A honeypot is a system or service designed to detect, deflect, or otherwise counteract attempts at unauthorized access or use of computer systems.

In other words, it’s a fake service, designed to look like a real service, to trick attackers and other bad actors looking to compromise systems. Attackers will perform scans, attempt exploits, and even upload malicious files to these honeypots in attempt to exploit them to gain access to underlying systems*.

(Hint: They’re not exploitable and the underlying system is safe.)

Step 1 - Sensor Network Design

In part 1 we discussed installation of T-Pot on a fresh linux install. Here we’re going to go over using Terraform to automate deploy and teardown network with a T-Pot instance.

Below is a network diagram of what we’re going to build and deploy.

Overall this is a simple deployment and looking at the things we must create we have:

  • 1 VPC
  • 3 Compute Instances
    • “sensor-asia”
    • “sensor-europe”
    • “Collector & Sensor” aka “The Collector”
  • 3 Firewall Rules
    • Allow internal traffic
    • Allow traffic to sensors
    • Allow traffic to collector

Two of the compute instances are identical, T-Pot honeypot sensor only (with some minor modifications). The third is a full T-Pot installation, all honeypots and ELK stack, The Collector.

To aggregate the events into The Collector the two sensor instances will use the existing logstash container to put events into The Collector’s elasticseach database. To achieve this we will have to make two changes to the system after T-Pot installation.

1. Edit sensor `logstash.conf` to point to The Collector’s `elasticsearch`
2. Edit The Collector `tpot.yml` to expose the  `elasticsearch` service

Step2 - Sensor Network Deployment!

To deploy our sensor network we’re going to use Terraform. This is something I covered in my DFIR Lab post, but to reiterate for the newcomers.


From the HashiCorp (vendor) docs

Terraform is a tool for building, changing, and versioning infrastructure safely and efficiently. Terraform can manage existing and popular service providers as well as custom in-house solutions.

Configuration files describe to Terraform the components needed to run a single application or your entire datacenter.

This means, terraform can create, modify, delete ANY resource outlined in the configuration file. The configuration file is the code that outlines all the infrastructure: network, compute, server less functions, and other vendor supplied resources.

With that out of the way, the important Terraform blocks we’re going to need for our sensor net are:

  • Provider
  • Create VPC
  • Create Instances
    • Startup Scripts
  • Create Firewall Rules

Connecting to the cloud provider

Provider blocks are needed to tell terraform which cloud provider to connect to and what credentials are we going to use to authenticate to that cloud service provider.

provider "google" {
  credentials = file("service_account_key.json")
  project = var.project_name      # honeypot-town
  region  = var.region_name       # us-east4
}

In this block we define our provider to be google and provide the credentials located in service_account_key.json. The project and region are specified in variables, which are assigned values in the file variables.tfvars.


Check out this post to read about service account impersonation so you don’t need to use and store service account keys.

Creating the project VPC

A Virtual Private Cloud (VPC) is your slice of the provider’s cloud infrastructure network. It’s where your resources are stored and isolated from other cloud users and their resources.


resource "google_compute_network" "vpc_network" {
  name                    = "honeypot_vpc"
  auto_create_subnetworks = "true"
}

Above we define a google_compute_network (VPC) which we will be able to find in our GCP console named honeypot_vpc. vpc_network is the unique variable name for referencing our VPC in our Terraform Configuration file.

To save us the effort of defining regional subnets we allow our provider to create them for us by setting auto_create_subnetworks to true. If we wanted to create and define our own subnets with specific IP ranges we would set the value to false and then google won’t auto provision them.

The instance to run T-Pot Sensors

Once the VPC is created we can create the 3 compute instances needed to run our network. The block needed to create the compute instances is the biggest, so let’s take a look at it and then break it down.

resource "google_compute_instance" "sensor-asia" {
  name         = "sensor-asia"

  machine_type = var.sensor_machine_type   # n1-standard-1 
  zone         =  "asia-east1-a"
  tags = ["sensor"] 

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
      size = 200
    }
  }

  network_interface {
    network = google_compute_network.vpc_network.self_link
    access_config { }
  }
   metadata_startup_script = var.sensor_startup_script
}

This block defines a google_compute_instance with the Terraform variable name sensor-asia . The new compute instance created is going to be named sensor-asia and its going to be a n1-standard-1 machine (one of many hardware specification types GCP provides to us, read the docs here).

A bootable disk is going to be made. It’s going to be 200 GB and will boot (a GCP image of) Debian 9 (“stretch”).

Our instance(s) will have a public IP address as well as a private one in the vpc_network we defined earlier. Since we used auto create subnets, the private IP will be within the CIDR range for the zone the instance is created in. You can see the default IP ranges google uses, here.

It will also have the network meta tag sensor and be located in the us-east4-c zone.

Finally, we declare there will be a startup script that will run every time the machine boots. This script is how we will install software and configure the instances programmatically rather than manually.

We’ll get into the startup scripts a bit further on, lets finish up the infrastructure deployment first.

The block above will be used to deploy our sensors and you will need one block for every sensor you want to deploy, with the appropriate declarations for where you want them deployed. You aren’t limited to running just 3 sensors, or to the regions/zones I use in this post; feel free to run as many or few sensors, wherever you want.

The Collector instance will use the same block except for some minor changes.

resource "google_compute_instance" "TheCollector" {
  name         = "TheCollector"

  machine_type = var.thecollector_machine_type   # n1-standard-2
  zone         = "us-central1-c"
  tags = ["admin"] 

  boot_disk {
    initialize_params {
      image = "debian-cloud/debian-9"
      size = 200
    }
  }

  network_interface {
    network = google_compute_network.vpc_network.self_link
	  network_ip = "10.128.0.42"
    access_config { }
  }
   metadata_startup_script = var.TheCollector_startup_script
}

There are a few changes when compared with the sensor block above.

  1. Both Terraform variable name and GCP name are set to TheCollector
  2. It uses a different machine type - specified in the variable.tfvars file
  3. This machine is in a different zone us-central1-c
  4. It has a network meta tag admin
  5. A static private IP address is being set*

*Note : We use a static private IP because our sensors need to know where to send their data, so we define it ahead of time.

The startup script used here will also be discussed later in the post.

firewall

A firewall is a network security tool to monitor and control the flow of traffic between two network segments based on predefined rules.

To allow access to our VPC and instances we are going to create the following rules

  1. Allow all internal private ip 10.0.0.0/8 traffic to TheCollector
  2. Allow all incoming TCP traffic to anything thats a sensor
  3. Allow all incoming TCP traffic to TheCollector except tcp/9200

Below is the firewall rule-block to ALLOW all TCP and ICMP (ping) traffic to all sensors.


resource "google_compute_firewall" "allow_sensor" {
  name    = "honeytown-ingress-allow-all-sensor"
  network = google_compute_network.vpc_network.self_link

  allow {
    protocol = "icmp"
  }

  allow {
    protocol = "tcp"
    # When no ports are specific, all ports are allowed.
	  # ports = ["80","443"]
  }

  target_tags = ["sensor"]
  source_ranges = ["0.0.0.0/0"]
}

Firewall rules have a few key components

  • Direction of connection - Ingress / Egress
  • Priority - higher priority (lowest priority number) rule whose other components match traffic is applied
  • Action to take, ALLOW / DENY
  • Source - Where the traffic is coming from
  • Target - Where the traffic is going Protocol - ICMP / TCP / UDP (+port numbers)

(GCP) Firewall rules can be applied to all instances in a VPC, to a metatag or service account; here we’re targeting the sensor tag we applied to our sensor network. Using metatag’s to target firewall rules is great because you don’t have to know a compute IP address in advance or write CIDR range specific rules, GCP takes care of this for us.

The block above can be used a model for the 2 other rules needed, this is something you can try to get some more hands on rule writing, otherwise the full file will be at the bottom of this post.

Checkpoint If you made it this far, we have created the structure of our honeypot sensor network.

We have a VPC network with at least 1 sensor and 1 collector, important to note, is also a sensor and 3 firewall rules to allow traffic to our instances.

Now we bootstrap our machines.

Startup Scripts

We have two distinct system configurations, TheCollector and Sensor. They are almost identical but have some differences. Let’s take a look at both and then talk about how to integrate them to our Terraform configuration .tf file.

TheCollector

if [[ ! -f /finished.flag ]]; then

# Update System and install packages
sudo apt-get update 
sudo apt-get install git docker-compose apt-transport-https ca-certificates curl gnupg-agent software-properties-common python3-pip -y  ;

# Add and install docker
$(curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -)  ;
sudo apt-key fingerprint 0EBFCD88 ;
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" ;
sudo apt-get update -y ;
sudo apt-get install docker-ce docker-ce-cli containerd.io -y ;

# Clone & Install T-Pot 
sudo git clone https://github.com/telekom-security/tpotce ;
cd tpotce/iso/installer/ ;
sudo cp tpot.conf.dist tpot.conf ;
sudo sed -i '715,727d' install.sh
sudo  ./install.sh --type=auto --conf=tpot.conf  ;
sudo rm /opt/tpot/etc/tpot.yml

# Get customized tpot.yml
sudo wget https://raw.githubusercontent.com/echohtp/Global-T-Pot/main/cnc_tpot.yml -P /opt/tpot/etc/ ; 

# Create flag so we know we finished 
sudo touch /finished.flag ;

# Restart machine
sudo reboot -n;
fi

Sensor

if [[ ! -f /finished.flag ]]; then

# Update System and install packages
sudo apt-get update;
sudo apt-get install git docker-compose apt-transport-https ca-certificates curl gnupg-agent software-properties-common -y;

# Add and install docker
$(curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -);
sudo apt-key fingerprint 0EBFCD88 ;
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable";
sudo apt-get update -y;
sudo apt-get install docker-ce docker-ce-cli containerd.io -y;

# Clone & Install T-Pot 
sudo git clone https://github.com/telekom-security/tpotce;
cd tpotce/iso/installer/;
sudo cp tpot.conf.dist tpot.conf;
sudo sed -i '715,727d' install.sh
sudo ./install.sh --type=auto --conf=tpot.conf;

# Get customized tpot.yml
sudo wget https://raw.githubusercontent.com/echohtp/Global-T-Pot/main/sensor_t.yml -P /opt/tpot/etc/ ;
sudo rm /opt/tpot/etc/tpot.yml
sudo mv /opt/tpot/etc/sensor_t.yml /opt/tpot/etc/tpot.yml

# Get customized logstash.conf
sudo wget https://raw.githubusercontent.com/echohtp/Global-T-Pot/main/logstash.conf -P /opt/tpot/etc/ ;

# Create flag so we know we finished 
sudo touch /finished.flag ;

# Restart machine	
sudo reboot -n
fi

I’ve run into some issues inlining bash files and debugging them using compute instances can be time intensive and frustrating. My buddy Matt offered me this really neat trick.

base64 encode the script, and have it decode on startup

This turns a multi line script to a one line bash command! Honestly this blew my mind and saved me a lot of future debugging time.

With the scripts above saved to their own files we use the base64 command to generate an encoded string.

➜  ~ base64 sensor.sh
aWYgW1sgISAtZiAvZmluaXNoZWQuZmxhZyBdXTsgdGhlbgpzdWRvIGFwdC1nZX
[...SNIP ...]
GMvIDsKc3VkbyB0b3VjaCAvZmluaXNoZWQuZmxhZyA7CnN1ZG8gcmVib290IC1

Now in our .tf file or variables.tfvars we can use the following one-liner

base64 -d aWYgW1sgISAtZiAvZmluaX...[SNIP]...gcmVib290IC1 | bash (Note this is snipped for ease of reading, the “one-liner” is one line, but is actually quite long. )

Using the base64 command with the decode flag -d the encoded script file is decoded and then passed to bash using a pipe | where it is then interpreted and executed.

👍🏼 Super cool way to get a large script down to a single line!👍🏼

With all our pieces in place: ✅ Infrastructure ✅ Compute instances ✅ Startup scripts ✅ Firewall rules

we’ll use Terraform to deploy it all.

First step is to check our config file to make sure it has no errors and will deploy according to our plan.

➜ terraform plan -var-file=variables.tfvars
Refreshing Terraform state in-memory prior to plan...

[... SNIP ...]

Plan: 8 to add, 0 to change, 0 to destroy.

------------------------------------------------------------------------

Then we apply it and answer yes when prompted. and after a few minutes we’ll be informed of it being complete.

➜ terraform apply -var-file=variables.tfvars

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
  + create

[... SNIP ...]

Plan: 8 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

[... SNIP ...]

Apply complete! Resources: 8 added, 0 changed, 0 destroyed.

Verify Global Deployment!

We can check the google cloud console to see our that instances have been created and were created in the specified zones.

At this point, get yourself a cup of coffee, the instances startup scripts will take 15+ minutes to set themselves up and start passing data to the collector.

The Collector is a mostly standard T-Pot installation so to access our dashboard we’ll hit the public facing IP on port 64297. Login with the default username and password defined in the default config installation file, if you don’t want to use the default username and password, modify your startup scripts to edit the file; this isn’t something I did and if you want to secure further secure your system you should. I should also mention I’ve had issues connecting to the server (due to the default T-Pot certificates) on chromium based browsers, so use Firefox.

This is the standard T-Pot interface, there are lots of tools included in the package but the first thing we’re going to want to do is see if logs are coming in from our sensors. To verify connectivity we’re going to click into Kibana and then into Discover (the second icon down, compass). From there we’re going to type hostname the Search field names box and click t-pot hostname as seen in the screenshot below.

As expected there are three hostnames listed, sensor-asia, sensor-europe, cnc.us-central.

For your own deployments you can place sensors in different zones and deploy as many or as few as you like. Be aware the most sensors you deploy in the cloud, the bigger your bill will become.

Wait & Analyze data

If you’re following along and made to this point you have now deployed a global honeypot sensor network! The sensors need no additional maintenance and will collect and aggregate data as long as they are powered on and internet connected.

In my own research I left a deployment running for 48 hours and here are some statistics from that slice of time.

The biggest attack source offenders were:

  • 🥇Vietnam (25%)
  • 🥈Russia(13%)
  • 🥉India (13%)

The most in use attack operating system

  • 🥇Unknown (51.9%)
  • 🥈Windows 7 or 8 (44.7%)
  • 🥉Linux 2.x (1.3%)

Service most under attack

  • 🥇SMB (79.1%)
  • 🥈HTTPS (11.9%)
  • 🥉HTTP (3.8%)

Attacker source IP address

  • 🥇5.188.86.172 (72,128 events)
  • 🥈5.188.62.11 (10,518 events)
  • 🥉5.182.39.62 (7,033 events)

Wrapping Up

Honeypots are a fantastic and low cost source of threat intelligence. They can be leveraged by independent researchers to provide new insights or large corporations to augment a maturing security program. I hope this post helped you learn something new and inspired you to deploy your own honey network!


Here’s the Terraform gist!

Subscribe to the cloud.weekly newsletter!

I write a weekly-ish newsletter on cloud, devops, and privacy called cloud.weekly!
It features the latest news, guides & tutorials and new open source projects. You can sign up via email below.