Deploying Web stacks DRY-ly with Ansible, Part 2: Users and Logins

 

In the last part we began by setting up the core infrastructure necessary to stand up a new server using automation and Continuous Integration. A Gitlab instance serves as the repository, with Gitlab CI connecting to the target server to deploy playbooks and execute Ansible. Our playbooks were surprisingly sparse due to the fact we relied on Ansible Roles to do most of the heavy lifting. Now, all we need to do is push code to our infrastructure repository, and CI will configure the server for us.

Nothing's worse than a soggy login

At this point, we've installed a lot of software and done some configuration, but what about user accounts and databases? Certainly, if it's bad practice to put those in a Drupal site's repository, it's bad to put them in our infrastructure repository aswell. Still, we want to minimize having to ever directly log in to the server at all. There's a lot of ways to solve this problem, but the most direct way available to me was to use Gitlab Secret Variables

Secret Variables are stored encrypted on the Gitlab server and associated with a single Gitlab project. When CI is run, those variables are exposed as environment variables only to the build's shell session. Since the environment variables aren't permanent, the attack surface only exists during a build. This makes them perfect for us to use in our Ansible playbooks to set logins and create databases. The UI varies a bit depending on which version of Gitlab you're using. At the time of this writing, the process looks like this:

  1. Login to Gitlab if you haven't already done so.
  2. Navigate to the target project.
  3. Open Settings > CI/CD.
  4. Scroll down to the Secret Variables section. 
  5. Use the provided Key and Value to enter the variable name and value.
  6. Click Add New Variable.

Since Secret Variables must belong to a specific project, the Key need only be unique within the project. Existing Secret Variables may be managed on the same page. For this project, I decided to create a unique Secure Variable for each username and password:

  • MYSERVER_MYSQL_DRUPAL8_USER
  • MYSERVER_MYSQL_DRUPAL8_PASS
  • MYSERVER_SHELL_DRUPAL8_USER
  • MYSERVER_SHELL_DRUPAL8_PASS

This approach makes it easy to reference individual values, but you spend a alot of time in the UI. In theory, you could put small pieces of XML or YAML as a Value, but I haven't tried it myself. 

Now that we have our variables defined, we need to get them into our Ansible playbook. We already know that when the CI is run, the Secret Variables are exposed as environment variables. The variable name is the same as the Key we entered in the Gitlab UI. So, all we need to do is look up the environment variable, and assign it to an Ansible variable. There's a few different ways to do this, but the easiest is to use the lookup() filter when assigning the variable:

variable_name: "{{ lookup('env', 'MY_SECRET_VAR_KEY') }}"

The lookup() filter can do several different things depending on the first parameter. To query environment variables we specify env, with the second parameter the environment variable name. Note that if you fail to create the Secret Variable, the playbook will fail when doing the lookup. To avoid that, you can chain the lookup() filter with a default() filter:

variable_name: "{{ lookup('env', 'MY_SECRET_VAR_KEY') | default('some default value') }}"

Wringing out our databases

Now that we know how to get the Secret Variables into Ansible, we can add them to our group_vars/myserver.yml. The geerlingguy.mysql role is capable of creating databases and users for us:

mysql_databases:
  - name: "myserver_drupal8"
    encoding: "latin1"
    collation: "latin1_general_ci"
mysql_users:
  - name: "{{ lookup('env', 'MYSERVER_MYSQL_DRUPAL8_USER') }}"
    host: "%"
    password: "{{ lookup('env', 'MYSERVER_MYSQL_DRUPAL8_PASS') }}"
    priv: "myserver_drupal8.*:ALL"

The above creates a single database named myserver_drupal8 and a single user using our Secret Variables. We also specify the database encoding and collation, as well as the user's allowed hosts and database privileges. What if we need to create more than one database or user? Notice something interesting about how mysql_databases and mysql_users variables are structured. Both take an array (or "list") of items, each identified by a hypen. Since indentation is significant in *.yml files, each item is actually a set of key/value pairs (a "dictionary"). If we need to create mutliple databases, we need only add a futher item:

mysql_databases:
  - name: "myserver_drupal8"
    encoding: "latin1"
    collation: "latin1_general_ci"
  - name: "myserver_legacy_database"
    encoding: "latin1"
    collation: "latin1_general_ci"

What if we want to grant permissions to multiple databases to a single user? We can do that too! After digging around a bit in the geerlingguy.mysql role, the priv item can accept multiple permission sets, delimited by a forward slash:

mysql_users:
  - name: "{{ lookup('env', 'MYSERVER_MYSQL_DRUPAL8_USER') }}"
    host: "%"
    password: "{{ lookup('env', 'MYSERVER_MYSQL_DRUPAL8_PASS') }}"
    priv: "myserver_legacy_database.*:ALL/myserver_drupal8.*:ALL"

Raincoats for your groups and users

Now we know how to create multiple databases and MySQL users as needed. What about UNIX users? Ideally, we want to create a separate system account for each site the server will host. The web server process would be able to read those files either by changing to that user, or by running as a dedicated Apache account with the same group. The geerlingguy.apache role doesn't make system accounts for us, so we need to do that ourselves. 

Ansible provides two modules for this purpose, group and user which create system groups and users respectively on UNIX systems. If we only planned on creating a handful of groups and users, we could call each of these modules directly for each group and user we want to create:

- name: Create server user groups.
  group:
    name: "myserver_group_name"
    state: present
  no_log: true
- name: Create user.
  user:
    name: "{{ item.name }}"
    password: "{{ lookup('env', 'MYSERVER_SHELL_DRUPAL8_PASS') | password_hash('sha512') }}"
    group: "{{ lookup('env', 'MYSERVER_SHELL_DRUPAL8_USER') }}"
    groups: "myserver_group_name"
    generate_ssh_key: yes
    ssh_key_bits: "4096"
    shell: "/bin/bash"
  no_log: true

The above creates one group, myserver_group_name, and one user using the Secret Variables we defined earlier. We actually define the user's primary group as the same as the username, with the custom group we created as an additional group. Since we want to be able to access the account via SSH, we also instruct the user module to create a strong (4096 bit) SSH key. Notice that we run the password through password_hash() first, as the user module will not encrypt the password for us. A special instruction we gave Ansible is no_log. This tells Ansible not to output the command contents to standard out when executing. This is essential when creating groups and users so the password is not exposed.

Avoiding getting drenched by variables

Sure, this works, but what happens if we need to add more groups or more users? Then we need to duplicate each group and user task in the playbook for each group and user we need to create. This can make for a very verbose playbook that can be difficult to debug later. We could eliminate the task duplication by using Ansible's with_items, but then we need to update the playbook each time:

- name: Create server user groups.
  group:
    name: "{{ item }}"
    state: present
  no_log: true
  with_items:
    - "myserver_group_name"
    - "another_group"
    - "yet_another_group"

That works too, but finding the right task in the playbook to edit each time you need to add a new group or user is still annoying and can create unintentional build failures. It'd be better not to never update the playbook at all, only the group vars! To do that, we need to create a data structure similar to the mysql_users or mysql_database role variables we defined earlier.

server_users:
  - name: "{{ lookup('env', 'MYSERVER_SHELL_DRUPAL8_USER') }}"
    password: "{{ lookup('env', 'MYSERVER_SHELL_DRUPAL8_PASS') }}"
    groups:
      - "www-data"

The new server_users variable is structured similarly to the mysql_users variable: it's a list of dictionary items, one for each user to create. I could have also created a separate variable for all groups to create, but I'd rather keep the group names with the user to avoid duplication or reference problems. With our data structure defined, we can now add some tasks to our main.yml playbook to create the groups and users using our fancy extendable data structure:

- name: Create server user groups.
  group:
    name: "{{ item.group | default(item.name) }}"
    state: present
  with_items: "{{ server_users }}"
  no_log: true
- name: Create server users.
  user:
    name: "{{ item.name }}"
    password: "{{ item.password | password_hash('sha512') }}"
    group: "{{ item.group | default(item.name) }}"
    groups: "{{ item.groups | default('') }}"
    generate_ssh_key: yes
    ssh_key_bits: "4096"
    shell: "/bin/bash"
  with_items: "{{ server_users }}"
  no_log: true

The first task creates server groups by looping through all items in the server_users. If the item in server_users does not have a group, the user's name is used instead. The second task is more straightforward. It also uses with_items to loop though server_users, using the user name and password. If no group is specifyed for the user, the user name is used. This seems like duplication, but Ansible's user module doesn't create any groups for us, even the user's default group. 

And that's it! If we want to specify multiple users, we can now just add an item under server_users in our group vars, and never touch the playbook at all.

Conclusion

In this part, we created custom databases, MySQL users, as well as system groups and users. Sure, it sounds like boring stuff that can be done much faster with a few shell commands, but now everything we do is tracked in git. When new databases or users are added, we'll know exactly who and when as it's all in the repository. Creating and managing infrastructure as code is a key pillar of DevOps, and we can do it all with a few pieces of off-the-shelf open source software!

Originally, I had intended both this post and the previous part to be a single post. While writing it, it became obvious that I didn't have enough time to complete it in one sitting. Furthermore, I felt I wasn't giving the implementation the discussion and detail it needed. As a result, I broke this post up into two parts. This was the end of what I had originally expected for the post, but there's still one major piece missing. The playbook creates a LAMP server with enough custom configuration to fulfill most needs, but it doesn't implement HTTPS. At the time of this writing, I still haven't figured out how to write the Ansible necessary to auto-provision a new certificate with Let's Encrypt, but I certainly would like to. If there's going to be a part three, that's what we'll cover.

Thanks to our sponsors!

This post was created with the support of my wonderful supporters on Patreon:

  • aesmael
  • Alina Mackenzie​
  • Chris Weber
  • ikit-claw
  • Karoly Negyesi
  • Sascha Grossenbacher

If you like this post, consider becoming a supporter at patreon.com/socketwench.

Thank you!!!