}}

Provisioning Confluent

Enterprise DC/OS Preview Updated: February 15, 2017

About provisioning Confluent with a service account

The ability to provision Confluent with a service account varies by security mode.

  • disabled: not possible
  • permissive: optional
  • strict: service cannot be installed

To increase the security of your cluster and conform to the principle of least privilege, we recommend provisioning Confluent with a service account in permissive mode. Otherwise, Confluent will use the default dcos_anonymous account to authenticate and the dcos_anonymous account has the superuser permission.

To set up a service account for Confluent, complete the following steps.

  1. Create a key pair.
  2. Create a service account.
  3. Create a service account secret.
  4. Provision the service account with the necessary permissions.
  5. Create a config.json file.

Note: We will use confluent-kafka-principal as the name of the service account, confluent-secret as the name of the secret, confluent-private-key.pem as the name of the file containing the private key, and confluent-public-key.pem as the name of the file containing the public key. If you stick to these names, you can copy and paste the command snippets more easily. However, feel free to change these names as desired. Just remember to also change the values in the code samples before issuing the commands.

Create a key pair

First, you’ll need to generate a 2048-bit RSA public-private key pair. While you can use any tool to accomplish this, the Enterprise DC/OS CLI is the most convenient because it returns the keys in the exact format required.

Prerequisite: You must have the DC/OS CLI installed and the Enterprise DC/OS CLI 0.4.14 or later installed.

  1. From a terminal prompt, use the following command to create a public-private key pair and save each value into a separate file within the current directory.
    $ dcos security org service-accounts keypair confluent-private-key.pem confluent-public-key.pem
    
  2. Type ls to view the two new files created by the command. You may also want to open the files themselves and verify their contents.

  3. Continue to the next section.

Create a service account

About creating a service account

Next, you must create a service account. This section describes how to use either the Enterprise DC/OS CLI or the web interface to accomplish this.

Using the Enterprise DC/OS CLI

Prerequisite: You must have the DC/OS CLI installed, the Enterprise DC/OS CLI 0.4.14 or later installed, and be logged in as a superuser via dcos auth login.

  1. Use the following command to create a new service account called confluent-kafka-principal containing the public key you just generated.
    $ dcos security org service-accounts create -p confluent-public-key.pem -d "Confluent service account" confluent-kafka-principal
    
  2. Verify your new service account using the following command.
    $ dcos security org service-accounts show confluent-kafka-principal
    
  3. Continue to Create a service account secret.

Using the web interface

  1. In the DC/OS web interface, navigate to the System -> Organization -> Service Accounts tab.

  2. Click New Service Account.

  3. Enter a description, the service account ID (confluent-kafka-principal), and the public key associated with the account. Simply copy the contents of the relevant public key file and paste it into the Public Key field.

  4. Continue to the next section.

Create a service account secret

About creating a service account secret

Next, you need to create a secret associated with the service account that contains the private key. This section describes how to use either the Enterprise DC/OS CLI or the web interface to accomplish this.

Using the Enterprise DC/OS CLI

Prerequisite: You must have the DC/OS CLI installed, the Enterprise DC/OS CLI 0.4.14 or later installed, and be logged in as a superuser via dcos auth login.

  1. Use the following command to create a new secret called confluent-secret containing the private key, the name of the service account, and other data.
    $ dcos security secrets create-sa-secret --strict confluent-private-key.pem confluent-kafka-principal confluent-secret
    
  2. Ensure the secret was created successfully with the following command.
    $ dcos security secrets list /
    
  3. If you have jq 1.5 or later installed, you can also use the following command to retrieve the secret and ensure that it contains the correct service account ID and private key.
    $ dcos security secrets get /confluent-secret --json | jq -r .value | jq
    
  4. Now that you have stored the private key in the Secret Store, we recommend deleting the private key file from your file system. This will prevent bad actors from using the private key to authenticate to DC/OS.
    $ rm -rf confluent-private-key.pem
    
  5. Continue to Provision the service account with permissions.

Using the web interface

  1. Log into the DC/OS web interface as a user with superuser privileges.

  2. Open the System -> Security tab.

  3. Click New Secret.

  4. Type confluent-secret into the ID field.

  5. Paste the following JSON into the Value field.

    {
      "scheme": "RS256",
      "uid": "confluent-kafka-principal",
      "private_key": "<private-key-value>",
      "login_endpoint": "https://master.mesos/acs/api/v1/auth/login"
    }
    
  6. Replace <private-key-value> with the value of the private key created in Create a key pair.

  7. Click Create. Your secret has been stored!

  8. Continue to the next section.

Provision the service account with permissions

About the permissions

In permissive mode, the Confluent service account does not need any permissions.

While Confluent cannot be deployed in strict security mode, we do plan to release a version that supports strict mode. If you plan to upgrade to strict at some point in the future, we recommending assigning the Confluent service account the permissions needed in strict mode to make the upgrade easier. The permissions will not have any effect until the cluster is in strict mode. If you plan to remain in permissive mode indefinitely, skip to Create a config.json file.

If you are in strict mode or want to be ready to upgrade to strict mode, continue to the next section.

Creating and assigning the permissions

With the following curl commands you can rapidly provision the Confluent service account with the permissions required in strict mode. These commands can be executed from outside of the cluster. All you will need is the DC/OS CLI installed. You must also log in via dcos auth login as a superuser.

Prerequisite: If your security mode is permissive or strict, you must follow the steps in Obtaining and passing the DC/OS certificate in curl requests before issuing the curl commands in this section. If your security mode is disabled, you must delete --cacert dcos-ca.crt from the commands before issuing them.

  1. Issue the following commands to create the permissions.

    Note: There is always a chance that the permission has already been added. If so, the API returns an informative message. Consider this a confirmation and continue to the next one.

    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:framework:role:confluent-kafka-role -d '{"description":"Controls the ability of confluent-kafka-role to register as a framework with the Mesos master"}' -H 'Content-Type: application/json'
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:reservation:role:confluent-kafka-role -d '{"description":"Controls the ability of confluent-kafka-role to reserve resources"}' -H 'Content-Type: application/json'
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:volume:role:confluent-kafka-role -d '{"description":"Controls the ability of confluent-kafka-role to access volumes"}' -H 'Content-Type: application/json'
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:reservation:principal:confluent-kafka-principal -d '{"description":"Controls the ability of confluent-kafka-principal to reserve resources"}' -H 'Content-Type: application/json'
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:volume:principal:confluent-kafka-principal -d '{"description":"Controls the ability of confluent-kafka-principal to access volumes"}' -H 'Content-Type: application/json'
    

    Important: Confluent generates its role name automatically by appending -role to the name value. By default, Confluent uses confluent-kafka as its name. The default role value will be confluent-kafka-role, as shown in the curl code samples. If you’re running more than one instance of Confluent, you will need to override the default name value and you’ll need to replace the instances of confluent-kafka-role throughout these curl samples with the correct role name. For example, if you change the name value to confluent-kafka2 for your second instance, you must replace each role value in the code samples to confluent-kafka2-role.

  2. Grant the permissions and the allowed actions to the service account using the following commands.

    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:framework:role:confluent-kafka-role/users/confluent-kafka-principal/create
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:reservation:role:confluent-kafka-role/users/confluent-kafka-principal/create
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:volume:role:confluent-kafka-role/users/confluent-kafka-principal/create
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:task:user:nobody/users/confluent-kafka-principal/create
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:reservation:principal:confluent-kafka-principal/users/confluent-kafka-principal/delete
    $ curl -X PUT --cacert dcos-ca.crt -H "Authorization: token=$(dcos config show core.dcos_acs_token)" $(dcos config show core.dcos_url)/acs/api/v1/acls/dcos:mesos:master:volume:principal:confluent-kafka-principal/users/confluent-kafka-principal/delete
    
  3. Continue to the next section.

Create a config.json file

If you have used all of the values shown in the previous sections, you can just copy and paste the following JSON into a new file and save it as config.json. Otherwise, change the values in the following JSON as appropriate.

{
  "service": {
    "principal": "confluent-kafka-principal",
    "secret_name": "confluent-secret"
  }
}

Continue to the next section.

Install Confluent

To install the service, use the following command.

$ dcos package install --options=config.json confluent-kafka

You can also provide the config.json file to someone else to install Confluent. Please see the documentation for more information about how to use JSON files to install services.