We Created a Fake Company Infrastructure for Student Club Members to Hack

Summary

As current leaders of Rochester Institute of Technology's Penetration Testing group (RVAPT), my colleague and I created a mini-competition that simulated a fake organization's IT infrastructure for the students to do a penetration testing on. With the help of my colleague and co-leader of RVAPT, Mohammed Alshehri(twitter), we decided to host this competition on his killer homelab infrastructure that he built from scratch.

The competition was created to provide a full penetration testing experience to college students, and to teach them what it is like to work with a real-world client - outside of their classroom environment. For two weeks, 13 underclassmen who have little to no offensive security experience participated. Students formed fictional security firms and did penetration testing on a fake web hosting company, RvaptHosting. This included Pre-Engagement, Enumeration, Exploitation, Post-Exploitation, and Reporting. While doing so, students wrote various documents, performed hands-on penetration testing, and presented their findings to their clients.  

This blog post explains the reason why I created this competition, how I created it, what kind of work the students had to go through, and what we learned along the way. Hopefully, this article could motivate other students to create interesting projects/competitions for their colleagues to enjoy and learn from.

Students' final penetration test reports are linked at Student Reports section as well. If you are interested, take a look!

Table of Contents

1. Background - RITSEC and RVAPT
2. Competition Scenario
3. Infrastructure
4. DMZ Network & Exploitation
5. Internal Network & Exploitation
6. Student Reports
7. Lessons Learned
8. Special Thanks

Disclaimer

All of the information and data represented in this document and RVAPT competition (hereafter referred to as “competition”) are not associated with real entities. All of the information was made-up; created in order to simulate the real-world penetration testing experience. RVAPT, its leaders, and its members do not have any association or relationship with any of the data within the competition.

Background - RITSEC & RVAPT

RITSEC is a student cybersecurity club at Rochester Institute of Technology (RIT). It has 100+ active members, 5 interest groups, 4 student-run competitions, and a 4-hour weekly meeting. There, RIT Vulnerability Assessment and Penetration Testing (RVAPT) is one of the interest groups that focuses on offensive security.

Ever since I became the co-leader of RVAPT, I wanted to provide my colleagues with a proper Offensive Security experience (more on here, link). This experience is not about just using a specific tool or "pwning" a machine. I wanted to provide a more well-rounded, real-world like offensive security experience. This would include working with clients, thinking about business values, setting up a clear goal, creating reports, and doing technical work, all at the same time.

And what better way is there than to create a fake company to do a penetration testing on? With the help of the co-leader Mohammed and his crazy home lab, we both decided to simulate a fake company's IT infrastructure on his lab, and let the students do a penetration test on it.

Competition Scenario

Fake company: Rvapthosting
Participant packet link
RvaptHosting's Request for Proposal link.

Students were asked to do penetration testing on a fake web hosting company, RvaptHosting. RvaptHosting is a company that has been in the web hosting industry for a long time, but have recently experienced various cyber attacks. Moreover, a (made-up) APT-Corona group was recently targeting web hosting companies, and RvaptHosting was concerned enough to request a penetration test to the security firms.

Students were expected to do an internal penetration test, with all the usual steps. Writing Response for Proposal and Request for Information, doing hands-on penetration testing, having a mid-engagement presentation, and delivering final report and presentation to their client.

In order to provide a full experience, I felt that going in-character was necessary. Going In-Character means both competition organizers and participants act as if they are playing a role. In this case, I acted as a CTO of RvaptHosting, and the students created their own security firms. I feel like this helped the students to get immersed in their roles and work.

To spice things up, I also added some ethics-related situations, Indicators of Compromise, and interaction with the client. This will be elaborated in the next section.

So, the scenario was all planned out. The students formed three security firms; Red-Force X, Black Hawk Security, and Starch Consulting. They were ready to compete.

Infrastructure

The overall infrastructure was very basic. It is two flat subnets in a VPN.  Sadly I was not able to simulate a real-world company using multiple subnets and such. This was mostly due to lack of time (3~4weeks), manpower, classes, competitions, mid-term exams, COVID-19, and much more. So this basic, yet fundamental, infrastructure is what we came up with.

Even though the infrastructure was basic, I tried to make all of the hosts have some kind of relationship with each other. This competition was not about hacking 10 boxes. It was supposed to be a simulation of a penetration test. So I tried to put relevant vulnerabilities and data in corresponding hosts. This will be explained further in the next section.

While the engagement scope included both the DMZ and the internal network, student teams were only introduced to the DMZ subnet first. The students were then to exploit the DMZ servers. After exploitation and post-exploitation, they were able to find enough hints to pivot into the internal network as well.  

DMZ Network

The DMZ infrastructure is where RvaptHosting's customers host their business; landing pages, eCommerce applications, blogs, and more. The network consisted of 4 different web servers and one management (MGMT) server which is used as a DMZ jumpbox for Rvapthosting's system administrators to use from the internal network.

One of my objectives in the DMZ was to install various types of vulnerable web applications. This included the typical LAMP stack, ReactJS application, dockerized containers, Perl-based web application, and more. The DMZ had a total of 7 vulnerable web applications and 15 vulnerabilities for the students to exploit.

Below are some of the "customers" of RvaptHosting. Again, every entity, organization, company, or group that was inside the competition infrastructure is fake, created by me. It was also disconnected from the internet behind VPN.

MISC - Suspicious Customer

One of my favorite fake customers was called Silkroad. As you have guessed, this customer was created after the infamous dark web website that was shut down in 2013. I wondered how students will react if they were to find illegal businesses during their penetration testing work. Will they ignore it, since it is none of their business? Will they put more attention since it is suspicious? Will they contact their point of contact right away?

I also created another short scenario that one of the employees in RvaptHosting was working for Silkroad, engaging in illegal businesses. Team 1 actually found this evidence and decided to report to the point of contact.

I was impressed when Team 1 - Red-Force X actually have found this evidence and reported back to RvaptHosting. Was this the right move, though? Personally, I am not sure. I heard conflicting opinions among professionals about whether penetration testers should behave like an "internet police" or not. Personally, I have no professional experience yet so I'm not 100% sure what are the protocols if this kind of situation were to happen. Nonetheless, Team 1 was bold enough to make a decision and contact its client.  

MISC - Indicators of Compromise

Another misc finding that I included was Indicators of Compromise throughout the servers in DMZ. One of RvaptHosting's initial goal of this penetration test was to make sure their network was safe from ATP-Corona. Turns out, it wasn't.

The bots that I actually used is from Yabnet, my personal project that I created for Red vs. Blue student competitions (github link).  The name "legion" was borrowed from the infamous Legion loader that was discovered in late 2019. The binary itself , however, is a simple Golang agent that calls back to a non-existing domain.

rvaptsupport@w1:~$ file /home/rvaptsupport/.legionminer_callbak 

/home/rvaptsupport/.legionminer_callbak: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, Go BuildID=4-STQTC99Tu0nzHwg-Gd/FI1397Ni8qbzAeb8z5Op/semgHYz_Vbyrs3cHKRn9/G-9tInaVi1gs4zU9S1tz, not stripped

Team 2 - Black Hawk Security was actually diligent enough to contact RvaptHosting about this issue. I was very pleased that Team 2 took IoC very seriously. They took the time to reverse it (a little bit), made sure the binary was suspicious and reported back to its client. Well done!

Exploitation - DMZ

I was going to provide a write-up on how to attack the DMZ, but that will make this blog post too lengthy and technical. Instead, I'll post a link to my write-up. Please be aware that the document is a single Attack Path out of multiple combinations. Overall, DMZ had around 15 vulnerabilities that could have been combined to create various attack paths.

Write-up Link

Let's focus on the contextual exploitation that some teams have found, and some have forgotten to find. One of the interesting things that I found was that some teams were treating this penetration test as a Pwn-to-Own. This means, if they were to find a single way to exploit something, they will often move on to the next target, without testing alternative ways to exploit the same service or application.

One example was the Webmin application that was running in MGMT host machine, 192.168.110.60. Webmin is a web application that allows users to remotely control specific hosts over the web. While all of the teams have found the weak credentials of the application, admin:webmin, some teams have not tested for the Webmin Remote Code Execution vulnerability.  They could have used a PoC, Metasploit module, or create a PoC on their own to test and verify the RCE. Instead, they were satisfied with getting inside and moved on to other targets.

Another example would be the sensitive database that could be found in the MGMT host machine. While the teams have done a great job reporting this finding, some teams failed to figure out a couple of things.

1. Mysql service itself was facing public. Considering this is a DMZ network, this could lead to a huge business impact.

2. Under PCI-DSS compliance, storage of Primary Account Number (PAN) is allowed, but it should be encrypted or masked.

3. Passwords were MD5 hashed.

4. The data contains Peronsally Identifiable Information (PII). Teams which found the data should have notified the Point of Contact.

5. The data is PII, so screenshots regarding this data in the final report should be redacted.

While there was a single team reached out to their client, not all of the teams have notified the Point of Contact. Moreover, not all of the teams have found the problems listed above as well. All of the teams have found the database and included in the final report though, so props to the teams!

Internal Network

The Internal infrastructure was a simple Windows Active Directory with an old abandoned FTP server. The internal network was able to communicate with the DMZ by using one of the system administrator's workstation.

In the previous section, the DMZ infrastructure was focused on web applications. The internal infrastructure, on the other hand, was focused on Active Directory environment and human errors that might happen while users or system administrators use their workstation.  Active Directory environment is also something that is hard to experience as students. So I wanted to provide the students with a simulated environment that they can learn lateral movement and post-exploitation.

Exploitation - Internal

Similar to the DMZ section, the write-up is linked below. If you are more of a technical person, take a look. Again, please beware the internal network had around 5~6 vulnerabilities, and the write-up only shows a small portion of them.

Internal Attack Path Link

While students were somewhat familiar with the classic web application exploitation, they were not familiar at all with the Windows Active Directory environment. Thus this portion of the competition wasn't so easy for them.

This section will go over some of the findings that they could have found along the way.  

Interesting Domain User Credential

One of the interesting findings was the domain user rvapthosting.com\josh 's credential. In the screenshot above, we see that josh domain user having local administrator rights on JOE's computer. It is known giving domain users local Administrator rights is not best practice. Moreover, in this case, josh has local administrator rights on different user's (JOE's) workstation. In-character, JOE is actually the CTO of RvaptHosting. This doesn't seem right.

Abandoned Servers and Asset Management

Another finding is an abandoned FTP server in the network. Just by changing josh's password from rvaptWinter2020! to rvaptWinter2015!,  it is possible to access the FTP server. Based on the FTP banner, the server has been EOL'ed since 2015 and was kicked out from the Active Directory. Indeed, if we perform some basic AD enumeration, we don't see the server on the list.

But the server is still up and running. Another question comes to mind; were the files on the server also been deleted using proper procedures?

No. Based on the file, it seems that one of the employees was storing CEO Bob's credentials inside a plaintext file. This finding with no password policy made it possible to login to Bob's workstation using the credential that was found 5 years ago.

Service Running with DA account

In Bob's workstation, Mimikatz shows that serviceadm's account credentials was stored in LSASS. What is this account? Since serviceadm is not a default account, let's do some enumeration.

Get-DomainGroupMember -Domain rvapthosting.com -Identity 'Domain Admins' | Select-Object membername

serviceadm is a Domain Administrator user! Behind the scene, the account was running a Windows Service on boot on Bob's workstation. Due to some misconfiguration, the serviceadm account actually has a password, and that is stored in LSASS every time Bob turns on his computer. With this information, it is possible to Pass-the-Hash serviceadm, and login to the Domain Controller.

Invoke-Mimikatz -Command '"sekurlsa::pth /user:serviceadm /domain:rvapthosting.com /ntlm:ec7153d05f54a2ce9c2dd16563f1672d /run:powershell.exe " '

That finishes off the Internal network portion as well. If the student teams were to be successful, they would have found sensitive PII data from the DMZ, compromise all machines in the DMZ and internal network, and become the Domain Administrator while taking over the CEO and CTO's personal workstation as well.

Timeline

Student Reports

Below links are the reports students wrote. If you are interested in talking to these students about this competition, please visit their linkedin links and send them a message.

Team 1 - Red-Force X (Best Documentation Award)

Final Report Link
Ryan Cervantes
Jorge Flores
Max Fusco

Team 2 - Black Hawk Security (Best Professionalism Award)

Final Report Link
Cullen Rezendes
Spencer Roth
Evan Mikulski
Elijah Heilman

Team 3 - Starch Consulting (Best Teamwork Award)

Final Report Link
Jacob Ruud
Emmanuel Adewale
Omar Aljaloud
Abdulmalik Banaser

Lessons Learned

This competition was a great experience for me. The project had a nice balance between technical sysadmin work, designing infrastructure, soft-skill work, and leadership. There were a lot of areas where I had to push my boundaries, as most of the work was something that I never done before.

Personally I have learned that building and running an infrastructure is not an easy thing.  Running a penetration test is not an easy thing. Communication between the client and the service provider is not an easy thing. Heck, there wasn't a single easy thing during the competition. I definitely started to respect all of the technical and non-technical professionals that are working in the industry.

Based on the competition debrief, the students really seemed to enjoy the overall experience. Even though the competition was not all about the technical "hackhackhack", they enjoyed some of the soft-skill related work they had to do.

So that about wrap things up. I hope my experience would motivate the readers to start up your own projects or competition. "Security Through Community" is a motto that we use in RITSEC, and I couldn't agree more. Good luck, and have fun!

Special Thanks

Thank you National CPTC - its black team, organizers, sponsors (IBM and a lot more) - for the inspiration, motivation, and fun experience. This mini-competition was heavily inspired by the National CPTC competition.

Thank you Mohammed - As a friend, colleague, co-leader of RVAPT, and infrastructure architect, this competition would not have happened without you.

Thank you alumni coaches - Scott Brink, Will Eatherly, Matt Brown. It really means a lot when alumni still keep in touch with current students.

Thank you Boan Project (조정원) - For inspiring me to start this competition.