What this is and isn't
This is a proof of concet for a shared hosting environment where users are allowed to execute arbitrary
code and bind their apps to all non-privileged ports in a secure manner. During the development of a new
version of Uberspace I was trying to come up with a solution to the following problems:
Current Uberspace systems are persistently checking for processes that use more than 600MB of memory for a longer period to ensure that there are no excessive noisy neighbors on the system that could limit the experience of other users on the system. These processes are then sent a SIGTERM and the owner of the process is informed via mail about the reasons for doing so. This approach doesn't work for I/O and CPU shares though.
This prototype demonstrates how to create unprivileged users with specified resource limitations by
creating a Control Group (this is why it's called cgroups) with pre-defined values for
Uberspace currently allows all users to bind to any unpriviled ports. This is a security risk as another user on the system can try to bind to a previously used port in case of a reboot or a crash of the service and collect all incoming traffic to this port or even impersonate the service, if it isn't properly secured, e.g. by using selfsigned certificates or no TLS at all.
I tried to leverage SELinux to assign ports to SELinux-users so that all users are still able to connect to assigned ports without being able to bind their own apps to any port that isn't assigned to them. According to my tests I was successful.
Note: To the best of my knowledge the scenario I described never happened in all of my 4 years at Uberspace.
Uberspace currently disables SELinux. Everyone is scared of the complexity of SELinux and therefore just disables SELinux. According to my own experience, documentation of SELinux is also pretty overwhelming, outdated and often distribution-vendor-§specific.
My approach is based on a blog post by Major Hayden.
This attempt should allow users to do anything that unprivileged users are do on a Linux system while limiting the scope of the impact of e.g. a local root exploit on a shared hosting system like Uberspace, as long as the attack isn't targeted against SELinux in the Linux kernel.
Why not use containers?
To the best of my knowledge, there is no way to allow unprivileged users to create containers without either adding them to a privileged group that is root-equivalent while also exposing control over all containers to any user on the system. Due to the fact that Uberspace is still primarily shared hosting and not a PAAS-solution, containers where not an option for the version of Uberspace that I was working on.
Will this be used for Uberspace?
I do not know if any of this will end up in the next version of Uberspace because I am no longer working for Uberspace. Please direct all questions regarding the new version of Uberspace to either email@example.com or to @ubernauten on Twitter.
Please consider this as a proof of concept. Do not use this in production without further testing!
We can confine users to only use specified resources.
Users created using
site.yml will be created with a limit of 512CPU shares (out of 1024, which equals to 50% of all cores) and 1GB of RAM (cumulated virtual memory of the user). The user variable defaults to
Test & Verify
Login is as our test user. Defaults to
$ ssh firstname.lastname@example.org -p 2222 -i .vagrant/machines/selinux/virtualbox/private_key
Run the following command as
$ stress --vm 10 --vm-bytes 120M --timeout 60s
Run the following command as user
vagrant. Login via
Pay close attention to the
Memory columns showing the Slice
/user.slice/user-1001.slice 14 327.5 276.3M - -
The CPU percentage should never grow beyond 512 while the Memory usage should never grow beyond 1GB of RAM.
Note: Both values can vary during verification.
You can also cross-check the configured limits using the following commands:
$ systemctl show -p CPUShares user-1001.slice CPUShares=512 $ systemctl show -p MemoryLimit user-1001.slice MemoryLimit=1073741824
SELinux Basic Usage
Our current default policy per user allows connections to all outgoing ports. The policy is defined in the template files
shelluser.te.j2. A detailed explanation of both template files will follow.
A user cannot bind to a local port, without first adding it to her user-specific
To ensure that a user cannot elevate their privileges by creating a Uberspace account like
unconfined, we add a prefix called
uberspace_. This also allows for easier management of Uberspace-user's ports.
semanage can modify user context modules and port types. Each modification of an existing user context is being merged into a SELinux-managed diff-table that is being applied to each user's existing policy. This means that we do not need to edit individual policies to add or remove ports. We can also modify a user's context without having to manually re-apply port modifications of a user's context.
In simpler words:
semanage keeps track of changes to individual user's security contexts outside of each policy template. Each change is applied to the base context, even if the context-module of the user is changed.
Add a port:
semanage port -a -t uberspace_poc_testuser_port_t -p $protocol $port
Delete a port:
semanage port -d -t uberspace_poc_testuser_port_t -p $protocol $port
Note: If a deleted port is still in use, the user will be able to use the port as long as her process is bound to the port.
List all ports enabled for a user:
semanage port --list | grep uberspace_poc_testuser_port_t
List all ports enabled for all Uberspace users:
semanage port --list | grep uberspace_
Test & Verify
Create a second user by replacing the user-variable in
site.yml. In this example, the user is
ubertest to bind to port 31337/UDP, which is the default port of
[vagrant@selinux ~]$ sudo semanage port -a -t uberspace_ubertest_port_t -p udp 31337
$ ssh email@example.com -p 2222 -i .vagrant/machines/selinux/virtualbox/private_key $ nc -u -l
Now log in as
[poc_testuser@selinux ~]$ nc -u -l Ncat: bind to :::31337: Permission denied. QUITTING.
As the example demonstrates, the user
poc_testuser cannot bind to the port that has just been
Now we send data via netcat from
[poc_testuser@selinux ~]$ echo "isthisevenworking?" > foo.txt && nc -u localhost 31337 < foo.txt [uberspace@selinux ~]$ nc -u -l isthisevenworking?
Note: As you can see, it is still possible to locally connect to a port that has been assigned to a user! If a user wants to ensure that access to her process is restricted to her account only, she must use a UNIX socket. Applications still need authentication when binding to a TCP/UDP-port, even if the port is not opened in iptables!
Binding to sockets is possible without any limitations.
Useful information about SELinux and its tools
NOTE: I haven't checked all of this in months. Please feel free to correct me wherever I am wrong!
New files and directories inherent the parent directory's SELinux type... tbc.
Identity: An identity is not the traditional UNIX UID. An SELinux identity can be assigned to one or more specific UNIX UIDs. The identity can be switched. tbc.
- Shows current user's SELinux context
- Show SELinux context of files or directories
zusermod -Z selinux_policy user
The following SELinux User Capabilities are available:
An overview of all SELinux-context associations with Linux accounts can be found in
- Files, processes, ports
- user:role:type:level(optional) Type Enforcement tbc.
- set Enforce/Permissive
- get current mode
semanage login -l
- show User/SELinux User configuration
semanage fcontext -l
- list available contexts, use grep to filter
- search policy types
- verbose information on policies
- reset security context of files
- change security context of files temporarily
- will be reset during a relabel of the filesystem (restorecon)
- import compiled policy module
- list active policy modules
- compare current security context with default security context
- New tool introduced in RHEL7. Allows for easier policy creation.