Python – Getting HTML with DOM

The other day I encountered a scenario where I needed to get HTML with Python, but only after Javascript had finished running. I accomplished this using the selenium driver.

  1. Download selenium with pip install selenium
  2. Download the driver for the browser you want to emulate. You can download them from this page. The driver must be in the PATH variable or you will need to specify the path in the constructor for the webdriver.
  3. Import selenium with from selenium import webdriver
  4. Now use the following code:
browser = webdriver.Chrome()
browser.get(raw_input("Enter URL: "))
html_source = browser.page_source

Note: If you did not put your driver in path, you have to call the constructor with browser = webdriver.Chrome(<PATH_TO_DRIVER_HERE>)

Find ASCII String in a Packet with Scapy

from scapy.all import *

def find_str(pkt):
    if IP in pkt:
        ip_src=pkt[IP].src
        ip_dst=pkt[IP].dst
    if TCP in pkt:
        tcp_sport=pkt[TCP].sport
        tcp_dport=pkt[TCP].dport

        #print " IP src " + str(ip_src) + " TCP sport " + str(tcp_sport)
        #print " IP dst " + str(ip_dst) + " TCP dport " + str(tcp_dport)

        if (pkt[TCP].dport == 80):
                if(str(pkt[TCP].payload).find("username") > 0):
                        print(pkt[TCP].payload)

# Filter out the packets we aren't interested in...
sniff(offline='unsecure_login_updated.pcap',filter="ip and host 192.168.1.3 and tcp",prn=find_str)

LSM, SELinux, Netlabel, and CIPSO

LSM, SELinux, Netlabel, and CIPSO

If you are new to CIPSO and SELinux, their relationship is confusing to say the least. I wanted to shed some light for those just setting out on the topic as to how the four of these things work together.

LSM

Original paper describing LSM: Linux Security Module Framework.

Within the context of SELinux and Netlabel you may hear the term LSM domain, LSM user, LSM xxxx. Early on in the development of Linux users realized discretionary access control (DAC) did not provide sufficient security for the system. While convenient and straightforward, simple problems existed. If a process running as root is compromised, essentially the entire system is compromised. Enter mandatory access control (MAC) implemented by LSM. I won’t cover MAC here, but it provides a solution to the problems presented by DAC. Practitioners of Linux began building kernel modules to implement MAC, but the issue was there were many and none were standardized. LSM provides a solution to this problem by offering a common framework for structuring and building kernel security modules. SELinux is one such implementation compliant with the LSM model. There are many others so when you read the term LSM xxxx, what it is saying is the implementation of LSM xxxx for your specific kernel security module.  For example, LSM domain in SELinux simply refers to SELinux Domains. The name doesn’t have to directly correspond, but it typically does.

SELinux

Secure Linux is a kernel security module which implements the Bell-LaPadula MAC model. SELinux is an LSM framework compliant Linux kernel security module. It provides mechanisms for controlling the way in which users, processes, and files within a system interact. Most relevant to this post, it allows administrators to classify files, processes, users, etc according to a security classification scheme. Ex: Top secret, secret, unclassified, etc.

Multi-Level Security

Built into SELinux is something called MLS (Multi-Level Security). This enables Linux to label files according to security sensitivities with s0 being the lowest classification and s15 being the highest. Within each security sensitivity are categories which may range from 0-1024. The security sensitivity and category are combined to create what is called a security level. It is the security level that CIPSO transports to other devices on the network.

Netlabel

Netlabel is a toolset for supporting network-based labeling within SELinux. The MLS portion of SELinux creates and attaches labels to different objects within the system. Netlabel provides the means to affix security related labeling to packets in a CIPSO format.

CIPSO

Commercial Internet Protocol Security Option (CIPSO) is a generic specification for adding security levels to network traffic. Technically, it was never made a standard and remains an IETF draft, but is the defacto industry standard. SELinux passes security classification labels to the network stack and those security classification labels are translated to the CIPSO protocol and then sent across the network to other MLS enabled systems.

Setting Up SELinux (A quickstart)

What This Tutorial Covers

The aim of this tutorial is to get the user up and running with SELinux quickly and provide additional resources for further research.

Tutorial performed on: CentOS-7 with kernel version 3.10.0-514.el7.x86_64

Installing SELinux

Note: For this tutorial, all commands are run as root.

Start by installing all necessary SELinux files:

yum install policycoreutils policycoreutils-python selinux-policy selinux-policy-targeted libselinux-utils setroubleshoot-server setools setools-console mcstrans netlabel_tools selinux-policy-mls

Configuring MLS

Enable MLS

For a more in depth explanation of MLS concepts see here.

Next we must enable MLS. SELinux enables MLS via the /etc/selinux/config file. We suggest users change the SELinux mode to permissive. SELinux enforcing mode blocks connections and may prevent the operating system from booting. Change the SELINUXTYPE to MLS.

vim /etc/selinux/config


# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=mls

Now we need to instruct the system to relabel all files according to a default labeling scheme. This is accomplished by instructing the system to autorelabel everything. Do this by adding the .autorelabel file to the root directory.

touch ./autorelabel

Now reboot the system. It may take a bit for the system to relabel everything. You should see the following message while it is relabeling:

Once the system reboots you can check to ensure it is running in permissive mode and  that MLS is enabled:

sestatus

SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: mls
Current mode: permissive
Mode from config file: permissive
Policy MLS status: enabled
Policy deny_unknown status: denied
Max kernel policy version: 28

Warning: Most setups have a number of errors generated when MLS is enabled. MLS likely blocks a number of critical processes. I suggest you resolve these errors before swapping from permissive to enforcing.

You can check the logs to see the activity SELinux blocks:

cat /var/log/audit/audit.log | grep SEL

If nothing appears when you run the command you are likely safe to swap to enforcing assuming the system has performed all of its expected functionality.

Configuring MLS Security Levels

To configure the security levels for files and processes edit the file /etc/selinux/mls/setrans.conf:

#
# Multi-Level Security translation table for SELinux
#
# Uncomment the following to disable translation libary
# disable=1
#
# Objects can be labeled with one of 16 levels and be categorized with 0-1023
# categories defined by the admin.
# Objects can be in more than one category at a time.
# Users can modify this table to translate the MLS labels for different purpose.
#
# Assumptions: using below MLS labels.
#  SystemLow
#  SystemHigh
#  Unclassified
#  Secret with compartments A and B.
#
# SystemLow and SystemHigh
s0=SystemLow
s15:c0.c1023=SystemHigh
s0-s15:c0.c1023=SystemLow-SystemHigh

# Unclassified level
s1=Unclassified

# Secret level with compartments
s2=Secret
s2:c0=A
s2:c1=B

# ranges for Unclassified
s0-s1=SystemLow-Unclassified
s1-s2=Unclassified-Secret
s1-s15:c0.c1023=Unclassified-SystemHigh

# ranges for Secret with compartments
s0-s2=SystemLow-Secret
s0-s2:c0=SystemLow-Secret:A
s0-s2:c1=SystemLow-Secret:B
s0-s2:c0,c1=SystemLow-Secret:AB
s1-s2:c0=Unclassified-Secret:A
s1-s2:c1=Unclassified-Secret:B
s1-s2:c0,c1=Unclassified-Secret:AB
s2-s2:c0=Secret-Secret:A
s2-s2:c1=Secret-Secret:B
s2-s2:c0,c1=Secret-Secret:AB
s2-s15:c0.c1023=Secret-SystemHigh
s2:c0-s2:c0,c1=Secret:A-Secret:AB
s2:c0-s15:c0.c1023=Secret:A-SystemHigh
s2:c1-s2:c0,c1=Secret:B-Secret:AB
s2:c1-s15:c0.c1023=Secret:B-SystemHigh
s2:c0,c1-s15:c0.c1023=Secret:AB-SystemHigh

For an in depth explanation of configuring the security levels see this power point. Page 42 includes a sample configuration for a simple US DOD style classification system. You can print a quick listing of the security levels with the chcat -L command.

Configuring Users and Files for MLS

Next we need to add a system user to the list of SELinux users. SELinux maps the system users to SELinux users. SELinux users map to roles. The roles define which users may access what processes. Processes have permissions to act on specific objects IE: files, directories, TCP sockets, etc. This tutorial does a good job of explaining these concepts. See the section titled “Some Basic Terminology”. Now we will add a user:

semanage login --add --seuser user_u grant

user_u is what is called a type in SELinux terminology. In this case we want to add a user of user_u type. grant is the system user we want to add, or at least, I want to add :-D.

To view the user mappings you can use the list command within the semanage tool.

semanage login -l

This mapping also shows us what MLS range users have access to. In this case, we can see the user grant (me) only has access to the s0 security sensitivity. We can edit a user’s security access levels with the following command:

semanage login –modify –seuser user_u –range s0-s2 grant

If user_u has not been configured, you will likely receive this error:

[root@localhost ~]# semanage login --modify --seuser user_u --range s2:c100 grant
libsemanage.validate_handler: MLS range s2:c100 for Unix user grant exceeds allowed range s0 for SELinux user user_u (No such file or directory).
libsemanage.validate_handler: seuser mapping [grant -> (user_u, s2:c100)] is invalid (No such file or directory).
libsemanage.dbase_llist_iterate: could not iterate over records (No such file or directory).
OSError: No such file or directory

What this is telling you is that the user_u SELinux user does not have access to either s1 or s2 or both (you don’t know which). To fix this you must grant the user_u SELinux user permissions to those security sensitivities first.

semanage user --modify --range s0-s2 user_u

Now we can rerun the semanage login command and it should go through.

Finally, we have to set the permissions for files within the file system. First, it is helpful to be able to check the permissions on the file system. In most Linux commands which support SELinux, the security permissions of an object are viewable with the -Z option. Ex: ps -Z, ls -Z, etc.

ls -Z

WARNING: SELinux permissions do not supersede the regular DACL permissions set by Linux. DACL permissions are checked first and only after that check does the system consult SELinux. This means if DACL denies a user access to a file and SELinux permits it, the user will not be able to access the file.

Right now we see that test_file has a permission level of s0. If we wanted to make it so that user grant could not access the file we could change the file’s security sensitivity to s3.

Note: In SELinux processes may write to their own security level or up and may read their security level and down. The process cannot read up or write down.

You can set the SELinux file permissions with the chcon utility.

chcon -l s3 test_file

Now we see the permissions of test_file require s3 access.

Conclusion

Hope this tutorial saves everyone some time! If you have questions feel free to comment.

Helpful Commands

Edit SELinux Configuration File

vim /etc/selinux/config

Check SELinux Status

sestatus

Check logs for SELinux activity:

cat /var/log/audit/audit.log | grep SEL

Configure MLS Security Levels

vim /etc/selinux/mls/setrans.conf

Print a listing of the security levels:

chcat -L

Add a User to SELinux

semanage login --add --seuser user_u grant

List SELinux User Mappings and Permissions

semanage login -l

Grant permissions to an SELinux user for a specific security sensitivity range:

semanage user --modify --range s0-s2 user_u

Grant permissions to a system user for a specific security sensitivity range:

semanage login --modify --seuser user_u --range s0-s2 grant

List file or process SELinux permissions:

ls -Z
ps -Z

Change the SELinux file permissions:

chcon -l s3 test_file

 

A Simple SSL Client and Server in Python

Server

import socket
from socket import AF_INET, SOCK_STREAM, SO_REUSEADDR, SOL_SOCKET, SHUT_RDWR
import ssl

KEYFILE = 'server_key.pem'
CERTFILE = 'server_cert.pem'

def echo_client(s):
    while True:
        data = s.recv(8192)
        print(data.decode("utf-8"))
        if data == b'':
            break
        s.send(b'This is a response.')
        print('Connection closed')
    s.close()

def echo_server(address):
    s = socket.socket(AF_INET, SOCK_STREAM)
    s.bind(address)
    s.listen(1)
    s.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)

    s_ssl = ssl.wrap_socket(s, keyfile=KEYFILE, certfile=CERTFILE, server_side=True)

    while True:
        try:
            (c,a) = s_ssl.accept()
            print('Got connection', c, a)
            echo_client(c)
        except socket.error as e:
            print('Error: {0}'.format(e))

echo_server((socket.gethostbyname('localhost'), 8082))

Client

import ssl

port = 8082

import socket, ssl

while True:

    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

    # Require a certificate from the server. We used a self-signed certificate
    # so here ca_certs must be the server certificate itself.
    ssl_sock = ssl.wrap_socket(s,cert_reqs=ssl.CERT_REQUIRED, ca_certs='server_cert.pem')

    ssl_sock.connect(('127.0.0.1', 8082))

    ssl_sock.write(str(input("Enter Something: ")).encode())

    ssl_sock.close()

Pentesting: Runas – How to elevate the same command prompt from user to admin in Windows

The search that’s more likely to bring you here might be something to the effect of: how to elevate the same command prompt from user to admin in Windows. For me, it was surprisingly annoying and I wasted a lot more time than I thought I would tracking down a solution to this particular problem.

It’s a deceptively simple task and the short answer to your question is: depending on the scenario, you can’t – but there are ways around it. As the title eludes, this is unfortunately not a post about how to elevate your privileges in the same command prompt legitimately, but how to do so in a pentesting scenario. (Although if you are a legit admin, you can set things up so you can do it with psexec.)

Scenario

You are pentesting something and you can get command line access to a server as a user. You have administrative credentials, but you only have access to your one user shell and you want to elevate to administrator/system.

How to Do It

Fortunately, it’s pretty straight forward. First thing you have to do is get an a meterpreter shell running on the computer. You may run into AV and if that’s the case you’ll need to obfuscate your meterpreter somehow. Fortunately this is pretty easy to do, something like Veil will do the trick, but really any non-Metasploit crypto/obfuscation typically works. All the AV vendors have signatures for the built in Metasploit stuff so it tends to be fairly ineffective.

Once you have your meterpreter session running you can use a module called post/windows/manage/run_as. The options will look something like this:

CMD <YOUR_METERPRETER_PAYLOAD> yes Command to execute
CMDOUT false yes Retrieve command output
DOMAIN workgroup yes Domain to login with
PASSWORD <ADMIN_PASSWORD> yes Password to login with
SESSION <YOUR_USER_METERPRETER_SESSION> yes The session to run this module on.
USER <ADMIN_USERNAME> yes Username to login with

Where CMD could be anything you want to run as the administrator, but I typically just rerun my meterpreter payload to upgrade it to admin level and go from there.

How to: Unicode in URL with Python 3

I found this to be much more difficult than I thought it would be. The solution was simple, but finding it was a bit of a pain. Here is the solution:

from urllib.parse import quote
from urllib.request import urlopen

url_string = "http://your_site_here" + quote(SOME-STRING-WITH-UNICODE)
html = urlopen(url_string).read().decode('utf-8')

Extremely simple, but it caused me a fair amount of headache to figure that out.

Common errors that might bring you to this post:

AttributeError: ‘bytes’ object has no attribute ‘timeout’
TypeError: Can’t convert ‘bytes’ object to str implicitly
UnicodeEncodeError: ‘ascii’ codec can’t encode character ‘\xf3’ in position 6: ordinal not in range(128)
TypeError: a bytes-like object is required, not ‘str’
TypeError: expected bytes-like object, not str

 

Fixing asm: Internal error: unbalanced parenthesis in operand 1

I got this error the other day when I was trying to compile the exploit code for the mremap exploit on Linux. While I have programmed a lot I have not spent an inordinate amount of time with gcc and this was something of a pain to solve so I thought I would share a general solution.

1.) Begin by getting a dump of the assembly code. You can do this with gcc -S -o <name_of_output> <name_of_input>

2.) Go to the line the compiler told you there was an issue with. Now you can see what the problem is. In my case it was that the assembler came up with: movl $((17 which obviously has missing parenthesis.

3.) You can either directly modify the assembly, or trace it back to the root file. In my case, it was inline assembly and doing a search on the corresponding function name led me to the offending line: ” movl $(“xstr(CLONEFL)”), %%ebx \n”. It was a macro, which looked fine. However you can view preprocessor output with the -E option. Examining the preprocessor output led me to the line: ”               movl $(“”(17|0x00004000|0x00000100)””), %%ebx           \n”. I’m not sure why that doesn’t work, but I simply changed it to ”               movl $0x4111, %%ebx                     \n” and it worked.

Hope this saves someone some time. Feel free to comment if you have more specific questions.

Connect GNS3 to ESXi

Introduction

As far as I can tell, there’s no great way to make this happen. I will explain, but to give you an image up front, below is a diagram of what I did. It may seem daunting at first, but I’ll explain as we go along.

gns3-esxi-diagram

Set Up Description

My GNS3 server is running on a VM on my ESXi host. In my case, I was running GNS3 on top of Ubuntu 15.10. My GNS3 server has four interfaces relevant to this problem. Interface eno16777984 connects from the VM to the default vSwitch0, which has access to the ESXi server’s one real network interface card. Interface eno33557248 connects to a second virtual switch I created, vSwitch1. This is the switch to which I connected the virtual machines I wanted to connect into my GNS3 topology. The interface tap1 is a loopback interface on the GNS3 server, which I used to connect into my GNS3 topology. Interface br0 bridges the tap1 interface to interface eno33557248. The bridge connects the virtual network created by vSwitch1 as bridged to my GNS3 topology.

Flow summary: VM hosted on ESXi -> vSwitch1 -> eno33557248 (GNS3 server)-> br0 (GNS3 server) -> tap1 (GNS3 server) -> GNS3 cloud

Apology: Sorry about the wonky interface names. Not sure why ESXi causes Ubuntu to generate such bizarre names.

Limitations

The limitation of this solution is that you may have to implement it multiple times if you want to connect different ESXi VMs into different locations within your GNS3 topology. For example, vSwitch1 could be used to service all the DMZ machines in a GNS3 topology. However, if you want to plug the ESXi VMs into a different location, say at the access layer, you will need to set up another iteration of this solution in its totality.

Configure ESXi Server

  1. Select your ESXi server, go to configuration->Networking
  2. Click Add Networking, Virtual Machine, Create a vSphere standard switch, label it and put it in a VLAN – I used VLAN 2, click finish
  3. On your GNS3 server, add a virtual NIC which is connected to your newly created vSwitch
  4. On your newly created vSwitch go to properties, highlight your newly created network (the one you named – not the one that says vSwitch), click edit, go to security and then click the checkbox next to promiscuous mode and change the setting to accept. This setting is not ideal and this technique should not be used in production networks. It essentially turns the vSwitch into a hub. I didn’t delve into depth on the issue, but I noticed vSwitch does not handled bridge traffic properly. It will forward layer two traffic, but layer 3. This setting is required for the bridge we create later on to work. As best as I can tell, it looks like vSwitch doesn’t learn the MAC addresses from the other network. So it doesn’t forward destination traffic properly.

esxi

My final configuration looked like this:

esxi-configuration

Configure GNS3 Server Interfaces

This is the tricky part of the operation. Credit goes to knowosielski for his post here for illustrating how to connect an interface into the GNS3 topology.

Create a Virtual Interface

  1. Create a shell script with the following content and put it in your location of choice (i.e. /scripts/<SCRIPT NAME>)

#!/bin/bash

#Set TAP1
tunctl -u husband

#Configure TAP1
ifconfig tap1 up

WARNING: Your tap interface may come up as tap0. That’s fine. When I set this up, I already had a tap0 so mine came up with tap1. If yours comes up as tap0, simply adjust the following steps accordingly.

  1. Modify the line “tunctl -u husband” and replace “husband” with your user name you want to have access to the interface.
  2. Save the script and make it executable with chmod +x <SCRIPT_NAME>
  3. Test the script by running it, then do an ifconfig and make sure tap1 is there. (Reminder, yours may come up as tap0, adjust steps accordingly if this is the case.)
  4. Modify “/etc/rc.local” to run this script every time the system starts. Add the line sudo <PATH_TO_SCRIPT>/<NAME_OF_SCRIPT> BEFORE the line exit 0. If you do not add the line before exit 0 it will not work. In mine I added the line sudo /home/husband/GNS3/script/tap
  5. Consider testing to make sure everything works by rebooting the system.

Create the Bridge Interface

  1. If you don’t already have them, run sudo apt-get install bridge-utils
  2. sudo vim /etc/network/interfaces add the line auto eno33557248 or whatever the name is of your VMs second interface. This should be the interface which resides in your newly created virtual ESXi virtual network which in my case was on vSwitch1.
  3. Now add the following lines:

# Bridge between tap1 and eno33557248
auto br0
iface br0 inet manual
bridge_ports tap1 eno33557248
bridge_stp off

  1. At this juncture, I strongly recommend you do a reboot and make sure that everything works at this point. If you skip this, troubleshooting down the line will probably be more challenging.

Configure GNS3

Now we’ll configure GNS3 itself. My setup was very simple for the sake of making sure everything works:

gns3_side

  1. In GNS3 click “Browse all devices” and drop the cloud into your topology
  2. Right click on the cloud and select configure
  3. Go to the tap tab, type tap1 (or whatever your tap interface is named), and click add

gns3

Now just drop a device in and connect it to the cloud and you should be up and running. I also tested this with the GNS3 ethernet switch and it worked fine. See screenshot below. This is a separate Ubuntu 15.10 server residing on my newly created ESXi vSwitch1, pinging through my GNS3 server and into the GNS3 topology.

gns32

This set up took me a really long time (especially that bit with promiscuous mode – that took forever to figure out). If you have any questions feel free to comment.