{"id":273,"date":"2014-02-28T20:12:00","date_gmt":"2014-02-28T20:12:00","guid":{"rendered":"https:\/\/questy.org\/?p=273"},"modified":"2024-09-26T19:08:30","modified_gmt":"2024-09-26T19:08:30","slug":"puppet-v-configuration-and-scaling","status":"publish","type":"post","link":"https:\/\/questy.org\/index.php\/2014\/02\/28\/puppet-v-configuration-and-scaling\/","title":{"rendered":"Puppet V – Configuration and Scaling"},"content":{"rendered":"\n

Introduction<\/strong><\/em><\/p>\n\n\n\n

The name of this portion of our tutorial was difficult to determine.  This is another set of configurations, but we will also be scaling Puppet to handle production quality traffic, be an external node classifier (ENC), have a backend database, employ an enterprise class web server, turn up a console\u2026  there\u2019s a lot to do.  We are indeed configuring the backend, but also scaling Puppet to handle your environment\u2026hence the name.<\/p>\n\n\n\n

This portion assumes you\u2019ve followed all previous tutorials from I-IV, have your certs signed and are complete and ready to go with Puppet \u201cas-is\u201d, you simply have not installed any of the following add-ons.  As mentioned last time, you could begin to write manifests and modules right now, using Puppet \u201cas-is\u201d, never utilizing any of the other features.  However, the \u201cout-of-the-box\u201d configuration of Puppet is not ready for enterprise use.  Perfect for a small development environment\u2026perhaps up to 25 hosts or so, the Puppet server as installed includes a small WEBRick server (ruby-based) and is not intended to handle large site traffic profiles.<\/p>\n\n\n\n

To make Puppet \u201centerprise-ready\u201d, we need to do a few things.<\/p>\n\n\n\n

    \n
  1. Install the Puppet Dashboard<\/li>\n\n\n\n
  2. Install the Puppet DB<\/li>\n\n\n\n
  3. Install passenger + Apache modules<\/li>\n\n\n\n
  4. Install MySQL<\/li>\n<\/ol>\n\n\n\n

    The dashboard gives you a GUI configuration mechanism as well as an external node classifier.  The PuppetDB is a centralized config storage mechanism for all node facts and configurations.  Passenger+Apache is the piece that replaces Puppet\u2019s WEBRick server, and MySQL holds the database for the dashboard.<\/p>\n\n\n\n

    This will be the longest portion of the tutorial series, and all the separate & individual pieces will be interdependent, requiring us to do all the work first with configuration testing at the end.  Let\u2019s get started.<\/p>\n\n\n\n

    Packages<\/strong><\/em><\/p>\n\n\n\n

    Early in tutorial I, I had you install a number of packages.  Had you stopped after installment IV, you\u2019d have had no need for a few of them, but I wanted all the packages to be on your system as prerequisites to avoid later installation headaches.  However, we do want to get the EPEL package set onto the system to add one prerequisite (and it\u2019s a darned fine repo to have, should you need it for other things)<\/p>\n\n\n\n

    To install EPEL, run the following command:<\/p>\n\n\n\n

    _**sudo rpm -ivh http:\/\/mirrors.kernel.org\/fedora-epel\/6\/i386\/epel-release-6-8.noarch.rpm**_<\/a><\/p>\n\n\n\n

    The \u201cExtra packages for Enterprise Linux\u201d (EPEL) set is an important addition to any server set.  Especially to install the package prerequisites we need.<\/p>\n\n\n\n

    Installing Packages<\/strong><\/em><\/p>\n\n\n\n

    Passenger:<\/strong><\/em> First, we will be installing Passenger.  The passenger packages are in their own repository, not hosted at Puppet Labs.  First, import the GPG key:<\/p>\n\n\n\n

    _**sudo rpm \u2013import http:\/\/passenger.stealthymonkeys.com\/RPM-GPG-KEY-stealthymonkeys.asc**_<\/a><\/p>\n\n\n\n

    And then install the passenger release repo:<\/p>\n\n\n\n

    _**sudo yum -y install http:\/\/passenger.stealthymonkeys.com\/rhel\/6\/passenger-release.noarch.rpm**_<\/a><\/p>\n\n\n\n

    Finally, install the Passenger Apache module to tie everything together:<\/p>\n\n\n\n

    sudo yum -y install mod_passenger<\/strong><\/em><\/p>\n\n\n\n

    Congratulations.  The groundwork for Passenger is now installed.<\/p>\n\n\n\n

    Dashboard <\/strong><\/em>Since we have done so much preparatory work, the dashboard install is quite simple:<\/p>\n\n\n\n

    sudo yum -y install puppet-dashboard<\/strong><\/em><\/p>\n\n\n\n

    Simple.<\/p>\n\n\n\n

    PuppetDB <\/strong>PuppetDB is installed a little differently, using Puppet itself to get and install the package:<\/p>\n\n\n\n

    sudo puppet resource package puppetdb ensure=latest<\/strong><\/em><\/p>\n\n\n\n

    This procedure takes a bit of time, but when complete, the PuppetDB is now installed.<\/p>\n\n\n\n

    MySQL <\/strong>MySQL is installed via the usual yum repos, but we will also turn it on and have it ready for use as well as create our users and remove unneeded and unsecured accounts for the system.<\/p>\n\n\n\n

    sudo yum -y install mysql-server <\/strong>sudo \/sbin\/chkconfig mysqld on<\/strong><\/em> sudo \/sbin\/service mysqld start<\/strong><\/em><\/p>\n\n\n\n

    Let\u2019s set the database root user\u2019s password:<\/p>\n\n\n\n

    ****\/usr\/bin\/mysqladmin -u root password \u2018\u2019<\/strong><\/em> \/usr\/bin\/mysqladmin -u root -h password \u2018\u2019 mysql -u root -p<\/strong><\/em><\/p>\n\n\n\n

    A few words here, for those of you unfamiliar with MySQL.  We are setting the root user\u2019s password to be able to administrate the database.  The simplest way to set up this initial security is using the \u201cmysqladmin\u201d tool provided by MySQL.  Note that when I use <> in these above, this is where your site-specific information comes into play.  For \\<FQDN>, for my example purposes I would replace this with puppet.example.com.  The password setting & changes, then, would look like so:<\/p>\n\n\n\n

    \/usr\/bin\/mysqladmin -u root password \u2018puppet\u2019 <\/strong>\/usr\/bin\/mysqladmin -u root -h puppet.example.com password \u2018puppet\u2019 \/usr\/bin\/mysql -u root -p<\/strong><\/em><\/p>\n\n\n\n

    I just wanted to clarify this for you in the event my use of <> and \u2018 \u2019 above caused any confusion.<\/p>\n\n\n\n

    Configuring MySQL<\/strong><\/em><\/p>\n\n\n\n

    Once you run the above commands, MySQL will prompt you for the password you just set.  Enter that password, and you will find yourself at a mysql prompt that looks like so:<\/p>\n\n\n\n

    mysql><\/strong><\/em><\/p>\n\n\n\n

    What this means is you have now logged into the MySQL database, and are ready to set it up for use.  Following I will list out all the commands you need to run in a set.  Note that these commands are each entered on a line and you press \u201c\u201d at the end of the line to enter the next command.  There is no output from MySQL when you enter these, so I\u2019ll enumerate them all together here for your convenience.<\/p>\n\n\n\n

    mysql> create database dashboard character set utf8;<\/strong><\/em>
    mysql> create user \u2018dashboard\u2019@\u2018localhost\u2019 identified by \u2018my_password’;<\/strong><\/em>
    mysql> create user \u2018dashboard\u2019@\u2018\u2019 identified by \u2018my_password’;<\/strong><\/em>
    mysql> grant all privileges on dashboard.* to \u2018dashboard\u2019@\u2018localhost\u2019;<\/strong><\/em>
    mysql> grant all privileges on dashboard.* to \u2018dashboard\u2019@\u2018\u2019;<\/strong><\/em>
    mysql> drop user \u2018\u2019@\u2018localhost\u2019;<\/strong><\/em>
    mysql> drop user \u2018\u2019@\u2018\u2019;<\/strong><\/em>
    mysql> drop database test;<\/strong><\/em>
    mysql> flush privileges;<\/strong><\/em>
    mysql> exit<\/strong><\/em><\/p>\n\n\n\n

    As before, replace with the fully qualified hostname for your server and \u2018my_password\u2019 with the password you wish to set for the dashboard user.  A few notes:<\/p>\n\n\n\n

      \n
    1. First, we created the dashboard database<\/li>\n\n\n\n
    2. Next, we created the dashboard user for connecting from the localhost name<\/li>\n\n\n\n
    3. Next, we created the dashboard user for connecting from the server FQDN<\/li>\n\n\n\n
    4. The next two, we grant the dashboard user rights to the whole database from either location<\/li>\n\n\n\n
    5. The following two lines delete the user \u2018\u2019 from the server (a null user w\/o a password)<\/li>\n\n\n\n
    6. Finally we drop the \u201ctest\u201d database, flush all our privilege tables (to take effect immediately) and exit MySQL.<\/li>\n<\/ol>\n\n\n\n

      The final steps in getting MySQL configured for production use is to tweak the settings in the database by editing the \/etc\/my.cnf file and restarting the database.  Open the \/etc\/my.cnf file and add a new line at the end of the file:<\/p>\n\n\n\n

      max_allowed_packet = 32M<\/strong><\/em><\/p>\n\n\n\n

      Save the file and then run<\/p>\n\n\n\n

      sudo \/sbin\/service mysqld restart<\/strong><\/em><\/p>\n\n\n\n

      for the changes to take effect.<\/p>\n\n\n\n

      Passenger <\/strong><\/em>The final piece is to get the appropriate passenger gems and the Apache module installed to handle Puppet Agent requests to the server.  Luckily, our previous prerequisite installs have made this easy for us.  First:<\/p>\n\n\n\n

      sudo gem install rack passenger<\/strong><\/em><\/p>\n\n\n\n

      When this is done, install the Apache module, following the prompts as follows:<\/p>\n\n\n\n

      ****sudo passenger-install-apache2-module Press Press At the end of the installation, Press <\/strong><\/em><\/p>\n\n\n\n

      If we\u2019ve done everything right up until this point, you should not need to supply any extra information, packages, or configuration, and only need to continue to press as listed above to complete the installation.<\/p>\n\n\n\n

      Now comes the time to configure the various pieces\u2026<\/p>\n\n\n\n

      Configuration<\/strong><\/em><\/p>\n\n\n\n

      Passenger <\/strong><\/em>First we need a number of directories and files to exist around the system, so let\u2019s put those in place by using the following commands:<\/p>\n\n\n\n

      sudo mkdir -p \/usr\/share\/puppet\/rack\/puppetmasterd
      <\/strong>sudo mkdir \/usr\/share\/puppet\/rack\/puppetmasterd\/public
      <\/strong><\/em>sudo mkdir \/usr\/share\/puppet\/rack\/puppetmasterd\/tmp
      sudo cp \/usr\/share\/puppet\/ext\/rack\/config.ru \/usr\/share\/puppet\/rack\/puppetmasterd
      sudo chown puppet:puppet \/usr\/share\/puppet\/rack\/puppetmasterd\/config.ru
      sudo chown puppet-dashboard:puppet-dashboard \/usr\/lib\/ruby\/gems\/1.8\/gems\/passenger-4.0.37\/buildout\/agents\/PassengerWatchdog<\/em><\/strong><\/p>\n\n\n\n

      Next, we need to configure the Puppet Dashboard to connect to its database, and setup the tablespaces for use:<\/p>\n\n\n\n

      cd \/usr\/share\/puppet-dashboard\/config
      <\/strong>edit database.yml
      Remove the last stanza of this file that refers to the \u201ctest\u201d database we removed above.
      For the Production and Development database stanzas, change the \u201cdatabase:\u201d line to read \u201cdashboard\u201d and the password line to contain your dashboard password so that it appears as follows:<\/strong><\/em><\/p>\n\n\n\n

      database: dashboard<\/strong><\/em>
      username: dashboard<\/strong><\/em>
      password: <\/strong><\/em><\/p>\n\n\n\n

      Next, prepare the database for use as follows:<\/p>\n\n\n\n

      cd \/usr\/share\/puppet-dashboard
      <\/strong>__sudo rake gems:refresh_specs
      <\/strong>rake RAILS_ENV=production db:migrate<\/strong><\/em><\/p>\n\n\n\n

      (Even though we\u2019ve reference the production and development databases in the database config above, we\u2019ll only be working in the production database in this tutorial)  <\/em><\/p>\n\n\n\n

      At this point, we should be ready to test the Dashboard configuration to ensure we\u2019re still on the right track.  Top do so, run the following:<\/p>\n\n\n\n

      cd \/usr\/share\/puppet-dashboard
      <\/strong>sudo .\/script\/server -e production<\/strong><\/em><\/p>\n\n\n\n

      Now, attempt to connect to the dashboard via web browser by pulling up the server at the following address:  http:\/\/<\/a>:3000.  If the dashboard displays correctly in your browser, we\u2019re ready to continue.<\/p>\n\n\n\n

      Press CTRL-C to exit the server.
      Configure Puppet for Dashboard<\/strong><\/em><\/p>\n\n\n\n

      While we have already configured the dashboard itself, we have not told Puppet the dashboard exists.  To do so, edit the \/etc\/puppet\/puppet.conf file and add the following.<\/p>\n\n\n\n

      In the [master] section of the puppet.conf, add the following lines:<\/p>\n\n\n\n

      Reporting<\/strong><\/em>
      reports = store,http<\/strong><\/em>
      _**reporturl =       :3000\/reports\/upload**_”>http:\/\/<FQDN>:3000\/reports\/upload**_<\/a><\/p>\n\n\n\n

      Node Classification (Using as an ENC)<\/strong><\/em>
      node_terminus = exec<\/strong><\/em>
      external_nodes = \/usr\/bin\/env PUPPET_DASHBOARD_URL=      http:\/\/localhost:3000<\/a> \/usr\/share\/puppet-dashboard\/bin\/external_node<\/strong><\/em><\/p>\n\n\n\n

      Exit the puppet.conf file, saving your changes and set permissions for following files like so:<\/p>\n\n\n\n

      sudo chown -R puppet-dashboard:puppet-dashboard \/usr\/share\/puppet-dashboard<\/strong><\/em>
      sudo \/sbin\/chkconfig puppet-dashboard-workers on<\/strong><\/em>
      sudo \/sbin\/service puppet-dashboard-workers start<\/strong><\/em><\/p>\n\n\n\n

      Apache<\/strong><\/em><\/p>\n\n\n\n

      Next, we need to configure the Apache web server to process requests being made by Puppet agents in your environment and hand them off to the Puppet server.  To do so, we need to create two files in the \/etc\/httpd\/conf.d location.  The passenger installation will have already created a passenger.conf there.  Just remove it before creating the following two files.<\/p>\n\n\n\n

      \/etc\/httpd\/conf.d\/dashboard.conf<\/strong><\/em><\/p>\n\n\n\n

      NOTE: The example file here is long lost.<\/em><\/p>\n\n\n\n

      And like it:<\/p>\n\n\n\n

      \/etc\/httpd\/conf.d\/passenger.conf<\/strong><\/em><\/p>\n\n\n\n

      NOTE: The example file here is long lost.<\/em><\/p>\n\n\n\n

       Starting Everything Up for Testing & Operation<\/strong><\/em><\/p>\n\n\n\n

      Once these configuration files are in place, it\u2019s time to test Apache\u2019s handoff to Puppet and to make a special SELinux module to allow Passenger handoffs to the various needed places in the filesystem.<\/p>\n\n\n\n

      First, make sure the puppetmaster process has been stopped:<\/p>\n\n\n\n

      **\/sbin\/service puppetmaster stop
      <\/em>\/sbin\/chckonfig puppetmaster off<\/strong>_<\/p>\n\n\n\n

      This assumes you\u2019ve run the procedures in the previous tutorials, including (especially) the certificate signing and exchange between master and agent.  If you\u2019ve done this, Passenger now has all the certs it needs to handle requests on behalf of Puppet, and no longer needs the Puppet server running.<\/p>\n\n\n\n

      Next, test the configuration, that you\u2019ve made no typos:<\/p>\n\n\n\n

      sudo \/sbin\/service httpd configtest<\/strong><\/em><\/p>\n\n\n\n

      If no errors are displayed, then at least <\/em>the syntax of your Apache configs are correct.  Now, to generate SELinux entries in the audit log to build a custom Passenger SELinux module, you need to start Apache:<\/p>\n\n\n\n

      sudo \/sbin\/service httpd start<\/strong><\/em><\/p>\n\n\n\n

      Turn off SELinux temporarily:<\/p>\n\n\n\n

      sudo setenforce 0<\/strong><\/em><\/p>\n\n\n\n

      Restart Apache to generate the log entries:<\/p>\n\n\n\n

      sudo \/sbin\/service httpd restart<\/strong><\/em><\/p>\n\n\n\n

      Test the Puppet dashboard in a browser by going to:<\/p>\n\n\n\n

      _**:3000**_”>http:\/\/<FQDN>:3000**_<\/a><\/p>\n\n\n\n

      What you have done is put output into the audit log that can be used as input to the audit2allow <\/em>tool to generate a policy file for import into SELinux to allow Passenger to do it\u2019s job.  To create that module, run:<\/p>\n\n\n\n

      grep httpd \/var\/log\/audit\/audit.log | audit2allow -M passenger<\/strong><\/em><\/p>\n\n\n\n

      This creates a new policy file for SELinux in your current working directory called \u201cpassenger.pp\u201d which you can now import into SELinux.  TO do so, simply import the module:<\/p>\n\n\n\n

      sudo semodule -i passenger.pp<\/strong><\/em><\/p>\n\n\n\n

      \u2026from the directory where the file resides (presumably your current working directory if you have not moved).<\/p>\n\n\n\n

      Finally, you re-enable SELinux, and begin to test your environment.<\/p>\n\n\n\n

      sudo setenforce 1<\/strong><\/em><\/p>\n\n\n\n

      Conclusion<\/strong><\/em><\/p>\n\n\n\n

      I know that\u2019s a lot for a single entry, but I wanted to make sure and get all the additional pieces installed and configured before starting to cover how each of the pieces work and interact.  I thought of making each piece its own article, but in every scenario I thought about, you ended up with a not-completey-configured (i.e., \u201cbroken\u201d) system.  I opted instead to do the complete configuration.<\/p>\n\n\n\n

      At this point, however, you should be all set and ready to go with Dashboard\/Passenger configured and being run by Apache.  You should have MySQL and PuppetDB installed and configured, handling their own individual tasks, and you should have Puppet Master and Agent installed (with one agent node) all configured to perform their duties as a Puppet Infrastructure.<\/p>\n","protected":false},"excerpt":{"rendered":"

      Introduction The name of this portion of our tutorial was difficult to determine.  This is another set of configurations, but we will also be scaling Puppet to handle production quality traffic, be an external node classifier (ENC), have a backend database, employ an enterprise class web server, turn up a console\u2026  there\u2019s a lot to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"advanced_seo_description":"","jetpack_seo_html_title":"","jetpack_seo_noindex":false,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":false,"jetpack_social_options":{"image_generator_settings":{"template":"highway","enabled":false},"version":2}},"categories":[4],"tags":[],"class_list":["post-273","post","type-post","status-publish","format-standard","hentry","category-puppet-education"],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_likes_enabled":true,"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/posts\/273","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/comments?post=273"}],"version-history":[{"count":2,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/posts\/273\/revisions"}],"predecessor-version":[{"id":275,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/posts\/273\/revisions\/275"}],"wp:attachment":[{"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/media?parent=273"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/categories?post=273"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/questy.org\/index.php\/wp-json\/wp\/v2\/tags?post=273"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}