blog.ryan.ec2016-11-02T23:30:46+00:00https://blog.ryan.ecRyan Cragunme@ryan.ecProtest Votes2016-11-02T00:00:00+00:00https://blog.ryan.ec/2016/11/02/protest-vote<p>As Election Day draws near I’ve noticed a stark difference between this election
and those of years past. During the Republican primaries support for Trump among
conservatives swirled in the 10-15% range. After he won he managed to secure the
votes of many in the remaining 85%. That’s a fairly remarkable feat considering that
the Republican party, who have long since proclaimed themselves the party of the
family, selected a racist, homophobic, xenophobic, sexist, ignoramus as their
candidate for president.</p>
<p>The liberal progressive reaction after Bernie Sanders dropped out of the Democratic
primary race has seemed to be the opposite. The media describe this lack of
support among younger voters as an “enthusiam gap” or “millenial cynicism” of
the political system and Hillary Clinton. It’s not hard to imagine that people
who came into adulthood during the great recession would be less than thrilled
at the prospect of a president who’s been bankrolled by Wall Street and who
supported the wars in Afghanistan, Iraq, Libya and Syria.</p>
<p>One interesting commonality of those who feel betrayed by their preferred
political party has been their preference of casting a protest vote for a third
party. This isn’t by any means a new phonomenon but I thought it might be best
to shed some light on why I think that protest votes are not only a distraction
but extremely dangerous.</p>
<h1 id="duvergers-law">Duverger’s Law</h1>
<p>In the US presidential elections we use a <a href="https://en.wikipedia.org/wiki/First-past-the-post_voting">first pass the post</a> plural
voting system. That is, one vote per voter for a single candidate. After all votes
are cast the majority <a href="https://en.wikipedia.org/wiki/United_States_presidential_election,_2000">usually</a> wins. A side effect of this type
of plural voting system is <a href="https://en.wikipedia.org/wiki/Duverger%27s_law">Duverger’s Law</a>, which states that these
types of plural voting systems favor a two party political system. We’ve seen
this in effect for quite a long time. You have to go back all the way to 1865,
when Andrew Johnson won the presidency as a National Union party member, to find
a president that wasn’t either a Republican or Democrat. The conclusion we can
draw here is that the next president of the United States will be Republican or
Democrat.</p>
<h1 id="lesser-of-two-evils">Lesser of Two Evils</h1>
<p>Even if you don’t want either of the major party candidates to win you must think
that one is worse that the other.</p>
<p>Because either a Republican or Democrat will win, by voting for a third party
candidate instead of voting against the worst major party candidate, you are
making it more likely that least desireable of the realistically electable
candidates will win. A vote for a third party does nothing but increase the
the likelyhood of the worst possible outcome.</p>
<h1 id="preferential-voting">Preferential Voting</h1>
<p>While our existing plural voting system will always favor two parties there could
be some light at the end of the tunnel. <a href="https://en.wikipedia.org/wiki/Instant-runoff_voting">Ranked-Choice voting</a> is a voting
system that allows you to rank your choice of candidates from most preferred to
least preferred. Votes are tallied in a way that effectively lets you vote for
your third party candidate while ensuring that the worst possible outcome is not
a threat. Some cities and states have adopted this style of voting already.</p>
<p>The only way out of our existing system is to change it. That’ll require a
constitutional amendment. Rather than focusing on casting a protest vote, vote for
the lesser of two evils and spend your time trying to elect local congressional
leaders that favor preferential voting.</p>
Automate publishing in the AWS Marketplace2015-08-19T00:00:00+00:00https://blog.ryan.ec/2015/08/19/automate-publishing-in-the-aws-marketplace<p>At CHEF we recently started publishing the Chef Server along with it’s Add-On’s into
the <a href="https://aws.amazon.com/marketplace/seller-profile/ref=srh_res_product_vendor?ie=UTF8&id=e7b7691e-634a-4d35-b729-a8b576175e8c">AWS Marketplace</a>. The documentation
provided by Amazon was mostly limited to general guidelines and the rules of publishing.
While it recommends building an automated process for publishing, it stops short
of prescribing a repeatable way to build, test and publish AMI’s into Marketplace.
What I’ve done is build a Chef resource that will allow you to easily automate
publishing your application into the AWS Marketplace. In this blog post I’ll
give an overview of the entire process of creating a cookbook for your application
and detail how to use the <code class="highlighter-rouge">marketplace_ami</code> resource to automate the building and publishing
in Marketplace.</p>
<h3 id="set-up-your-workstation">Set up your workstation</h3>
<p>Before we begin you’ll need to install and configure the <a href="https://downloads.chef.io/chef-dk">Chef Development Kit</a>
and the <a href="http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-set-up.html">AWS command line tools</a>. It’s not required, but you’ll probably
want to install <a href="https://www.virtualbox.org/wiki/Downloads">Virtualbox</a> and <a href="https://www.vagrantup.com/downloads.html">Vagrant</a> for local testing.
You can use Ec2 for testing as well.</p>
<h3 id="prepare-an-application-cookbook">Prepare an application cookbook</h3>
<p>Before you can automate the publishing of your application you first need to automate the
the installation and configuration of it. In this section I’ll go
through the process of writing a basic cookbook that will act as our app
in this tutorial.</p>
<p>If you’re already familiar with and/or have a cookbook for automating your application
you should skip ahead to the <a href="#publish-your-application">publishing section</a>.</p>
<p>ChefDK comes bundled with several command line utilities to assist in building and
testing Chef cookbooks. Here we’ll use the <code class="highlighter-rouge">chef</code> command to create a repository for your
chef policy and create a skeleton cookbook for the demo application
called <code class="highlighter-rouge">jackfruit</code> that we’ll eventually publish to the AWS Marketplace.</p>
<ul>
<li>
<p>Create a chef-repo and application cookbook</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> <span class="nb">cd</span> ~
<span class="nv">$:</span> chef generate repo chef-repo
<span class="nv">$:</span> <span class="nb">cd </span>chef-repo/cookbooks
<span class="nv">$:</span> chef generate cookbook jackfruit
</code></pre>
</div>
</li>
</ul>
<p>For the sake of brevity we’ll skip over the in’s and out’s of writing fully tested
idiomatic Chef cookbooks. Instead, we’ll limit the scope of this tutorial
to the task at hand. There are lots of other great resources for <a href="http://learn.chef.io/">learning Chef</a>
that I’d recommend going through if you’re new to using it.</p>
<ul>
<li>
<p>Create a recipe that will set up Apache and serve our application</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/recipes/default.rb</span>
<span class="n">include_recipe</span> <span class="s1">'apt'</span>
<span class="n">package</span> <span class="s1">'apache2'</span> <span class="k">do</span>
<span class="n">action</span> <span class="ss">:install</span>
<span class="k">end</span>
<span class="n">template</span> <span class="s1">'/var/www/htmp/index.html'</span> <span class="k">do</span>
<span class="n">source</span> <span class="s1">'index.html.erb'</span>
<span class="n">variables</span><span class="p">(</span>
<span class="ss">app: </span><span class="s1">'jackfruit'</span><span class="p">,</span>
<span class="ss">message: </span><span class="n">node</span><span class="p">[</span><span class="s1">'jackfruit'</span><span class="p">][</span><span class="s1">'message'</span><span class="p">]</span>
<span class="p">)</span>
<span class="n">notifies</span> <span class="ss">:restart</span><span class="p">,</span> <span class="s1">'service[apache2]'</span>
<span class="n">action</span> <span class="ss">:create</span>
<span class="k">end</span>
<span class="n">service</span> <span class="s1">'apache2'</span> <span class="k">do</span>
<span class="n">action</span> <span class="ss">:start</span>
<span class="k">end</span>
</code></pre>
</div>
</li>
<li>
<p>Add ‘message’ to the node attributes</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/attributes/default.rb</span>
<span class="n">default</span><span class="p">[</span><span class="s1">'jackfruit'</span><span class="p">][</span><span class="s1">'message'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'Welcome to the Jackfruit app on the AWS Marketplace!'</span>
</code></pre>
</div>
</li>
<li>
<p>Create our index.html.erb template</p>
<div class="language-erb highlighter-rouge"><pre class="highlight"><code># jackfruit/templates/default/index.html.erb
<span class="nt"><html></span>
<span class="nt"><head></span>
<span class="nt"><title></span><span class="cp"><%=</span> <span class="vi">@app</span> <span class="cp">%></span><span class="nt"></title></span>
<span class="nt"></head></span>
<span class="nt"><body</span> <span class="na">bgcolor=</span><span class="s">white</span><span class="nt">></span>
<span class="nt"><h1></span>Welcome to <span class="cp"><%=</span> <span class="n">app</span> <span class="cp">%></span>!<span class="nt"></h1></span>
<span class="nt"><p></span><span class="cp"><%=</span> <span class="vi">@message</span> <span class="cp">%></span><span class="nt"></p></span>
<span class="nt"></body></span>
<span class="nt"></html></span>
</code></pre>
</div>
</li>
<li>
<p>Update the metadata to depend on the ‘apt’ recipe which ensures that our apt
mirrors are up to date when chef installs packages.</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/metadata.rb</span>
<span class="nb">name</span> <span class="s1">'jackfruit'</span>
<span class="n">maintainer</span> <span class="s1">'ryan'</span>
<span class="n">maintainer_email</span> <span class="s1">'ryan@chef.io'</span>
<span class="n">license</span> <span class="s1">'ApacheV2'</span>
<span class="n">description</span> <span class="s1">'Installs/Configures jackfruit'</span>
<span class="n">long_description</span> <span class="s1">'Installs/Configures jackfruit'</span>
<span class="n">version</span> <span class="s1">'0.1.0'</span>
<span class="n">depends</span> <span class="s1">'apt'</span>
</code></pre>
</div>
</li>
<li>
<p>Configure test-kitchen to forward port 80 from the VM to our workstation on 8080.</p>
<div class="language-yaml highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/.kitchen.yml</span>
<span class="nn">---</span>
<span class="s">driver</span><span class="pi">:</span>
<span class="s">name</span><span class="pi">:</span> <span class="s">vagrant</span>
<span class="s">provisioner</span><span class="pi">:</span>
<span class="s">name</span><span class="pi">:</span> <span class="s">chef_zero</span>
<span class="s">platforms</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">name</span><span class="pi">:</span> <span class="s">ubuntu-14.04</span>
<span class="s">driver_plugin</span><span class="pi">:</span> <span class="s">vagrant</span>
<span class="s">driver_config</span><span class="pi">:</span>
<span class="s">network</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">[</span><span class="s2">"</span><span class="s">forwarded_port"</span><span class="pi">,</span> <span class="pi">{</span><span class="nv">guest</span><span class="pi">:</span> <span class="nv">80</span><span class="pi">,</span> <span class="nv">host</span><span class="pi">:</span> <span class="nv">8080</span><span class="pi">,</span> <span class="nv">auto_correct</span><span class="pi">:</span> <span class="nv">true</span><span class="pi">}]</span>
<span class="pi">-</span> <span class="s">name</span><span class="pi">:</span> <span class="s">centos-7.1</span>
<span class="s">suites</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">name</span><span class="pi">:</span> <span class="s">default</span>
<span class="s">run_list</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">recipe[jackfruit::default]</span>
<span class="s">attributes</span><span class="pi">:</span>
</code></pre>
</div>
</li>
<li>
<p>Use test-kitchen to converge the VM and manually verify the output. As you get more
familiar with Chef and test-kitchen you will want to automate this manual verification
using serverspec and test-kitchen suites. If you opted to not install vagrant
and virtualbox you could use the AWS test-kitchen driver in this step.</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> <span class="nb">cd</span> ~/chef-repo/cookbooks/jackfruit
<span class="nv">$:</span> kitchen converge
</code></pre>
</div>
</li>
</ul>
<p>You should now be able to visit localhost in your browser and see our application.
Be careful to watch the test-kitchen output as it will automatically fix port
collisions, as it did in my case.</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code> <span class="o">==</span>> default: Fixed port collision <span class="k">for </span>80 <span class="o">=</span>> 8080. Now on port 2204.
</code></pre>
</div>
<p><img src="/images/jackfruit.png" align="middle" /></p>
<p>Now that our demo application is automated we’ll destroy the VM and move on.</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> kitchen destroy
</code></pre>
</div>
<h3 id="publish-your-application">Publish your application</h3>
<p>For our publishing recipe we’ll use the <a href="https://github.com/chef-partners/marketplace_ami">marketplace_ami</a> resource
to provision a new EC2 instance, converge the jackfruit recipe, create and register
an AMI and share it with AWS Marketplace.</p>
<p>Optionally, you can enable a security recipe that will remove sensitive data by setting
<code class="highlighter-rouge">security</code> parameter to <code class="highlighter-rouge">true</code>. Also available is a chef-client audit mode
recipe that will audit the instance for known AWS security policies. You can
enable it by setting the <code class="highlighter-rouge">audit</code> parameter to <code class="highlighter-rouge">true</code>. Detailed information about
the resource can be found in the projects <a href="https://github.com/chef-partners/marketplace_ami">README on github</a>.</p>
<ul>
<li>
<p>Update the jackfruit cookbook dependencies</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/metadata.rb</span>
<span class="nb">name</span> <span class="s1">'jackfruit'</span>
<span class="n">maintainer</span> <span class="s1">'ryan'</span>
<span class="n">version</span> <span class="s1">'0.1.0'</span>
<span class="p">.</span><span class="nf">.</span><span class="p">.</span>
<span class="nf">depends</span> <span class="s1">'apt'</span>
<span class="n">depends</span> <span class="s1">'marketplace_ami'</span>
</code></pre>
</div>
</li>
<li>
<p>Create a publishing recipe</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># jackfruit/recipes/publisher.rb</span>
<span class="n">marketplace_ami</span> <span class="s1">'jackfruit-demo'</span> <span class="k">do</span>
<span class="n">instance_type</span> <span class="s1">'t2.medium'</span>
<span class="n">source_image_id</span> <span class="s1">'ami-123456'</span>
<span class="n">ssh_keyname</span> <span class="s1">'your_ssh_key'</span>
<span class="n">ssh_keypath</span> <span class="s1">'~/.aws/your_ssh_key.pem'</span>
<span class="n">ssh_username</span> <span class="s1">'ec2-user'</span>
<span class="n">product_code</span> <span class="s1">'123799879'</span>
<span class="n">security</span> <span class="kp">false</span>
<span class="n">audit</span> <span class="kp">false</span>
<span class="n">recipe</span> <span class="s1">'jackfruit::default'</span>
<span class="n">attribute</span> <span class="sx">%w(jackfruit message)</span><span class="p">,</span> <span class="s1">'Awesome new content!'</span>
<span class="n">action</span> <span class="ss">:create</span>
<span class="k">end</span>
</code></pre>
</div>
</li>
</ul>
<p>In the example I used several but not of the <code class="highlighter-rouge">marketplace_image</code> resource’s parameters.
Feel free to change the values to
reflect what you need. If you omit parameters, the resource
and sub-resources will attempt to guess sane values but I’d recommend being as
explicit as possible to ensure a consistent and repeatable build.
In the example I’ve disabled the security and auditing checks because
our demo cookbook will fail the audit. This isn’t a problem for the demo,
but when you’re publishing your application you’ll want to enable the audit during
build time and update your application cookbook until the audit passes. This
will help to ensure that your image will pass the AWS security scanning.</p>
<ul>
<li>
<p>Create a Berksfile in your chef-repo that points to your application cookbook</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># ~/chef-repo/Berksfile</span>
<span class="n">source</span> <span class="s1">'https://supermarket.chef.io'</span>
<span class="n">cookbook</span> <span class="s1">'jackfruit'</span><span class="p">,</span> <span class="ss">path: </span><span class="s1">'~/chef-repo/cookbooks/jackfruit'</span>
</code></pre>
</div>
</li>
<li>
<p>Now vendor all of the required cookbooks into your chef-repo</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> <span class="nb">cd</span> ~/chef-repo
<span class="nv">$:</span> berks vendor ~/chef-repo/cookbooks
</code></pre>
</div>
</li>
<li>
<p>Converge the the publisher recipe (output trimmed for clarity)</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> <span class="nb">cd</span> ~/chef-repo
<span class="nv">$:</span> chef-client -z -o jackfruit::publisher
Starting Chef Client, version 12.4.1
<span class="o">[</span>2015-08-19T15:10:07-07:00] INFO: Chef 12.4.1
<span class="o">[</span>2015-08-19T15:10:09-07:00] INFO: Run List expands to <span class="o">[</span>jackfruit::publisher]
resolving cookbooks <span class="k">for </span>run list: <span class="o">[</span><span class="s2">"jackfruit::publisher"</span><span class="o">]</span>
- Create jackfruit-demo with AMI ami-0372b468 <span class="k">in </span>us-east-1[2015-08-19T15:10:17-07:00] INFO: Processing chef_node[jackfruit-demo] action create <span class="o">(</span>basic_chef_client::block line 57<span class="o">)</span>
- jackfruit-demo is now ready[2015-08-19T15:10:50-07:00] INFO: <span class="o">[</span>AWS EC2 200 0.215999 0 retries] describe_instances<span class="o">(</span>:filters<span class="o">=</span>>[<span class="o">{</span>:name<span class="o">=</span>><span class="s2">"instance-id"</span>,:values<span class="o">=</span>>[<span class="s2">"i-0779e4ac"</span><span class="o">]}])</span>
- jackfruit-demo is now connectable[2015-08-19T15:13:04-07:00] INFO: Reading key ryan from file /Users/ryan/.chef/keys/ryan
- generate private key <span class="o">(</span>2048 bits<span class="o">)[</span>2015-08-19T15:13:07-07:00] INFO: Executing sudo ls -d /etc/chef/client.pem on ec2-user@52.2.69.39
- write file /etc/chef/client.pem on jackfruit-demo[2015-08-19T15:13:08-07:00] INFO: Processing chef_client[jackfruit-demo] action create <span class="o">(</span>basic_chef_client::block line 132<span class="o">)</span>
- write file /etc/chef/ohai/hints/ec2.json on jackfruit-demo[2015-08-19T15:13:10-07:00] INFO: Executing sudo ls -d /etc/chef/client.rb on ec2-user@52.2.69.39
- write file /etc/chef/client.rb on jackfruit-demo[2015-08-19T15:13:11-07:00] INFO: Executing sudo chef-client -v on ec2-user@52.2.69.39
- run <span class="s1">'bash -c '</span> bash /tmp/chef-install.sh -v 12.4.1<span class="s1">''</span> on jackfruit-demo[2015-08-19T15:22:21-07:00] INFO: Processing chef_node[jackfruit-demo] action create <span class="o">(</span>basic_chef_client::block line 57<span class="o">)</span>
- update run_list from <span class="o">[</span><span class="s2">"recipe[jackfruit::default]"</span>, <span class="s2">"recipe[marketplace_ami::_securitycontrols]"</span><span class="o">]</span> to <span class="o">[</span><span class="s2">"recipe[jackfruit::default]"</span>, <span class="s2">"recipe[jackfruit::default]"</span>, <span class="s2">"recipe[marketplace_ami::_sec
[jackfruit-demo] [2015-08-19T22:22:24+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.4.1
[2015-08-19T22:22:24+00:00] INFO: *** Chef 12.4.1 ***
[2015-08-19T22:22:26+00:00] INFO: Run List expands to [jackfruit::default, marketplace_ami::_security_controls]
resolving cookbooks for run list: ["</span>jackfruit::default<span class="s2">", "</span>marketplace_ami::_security_controls<span class="s2">"]
- chef-sugar
Compiling Cookbooks...
Converging 2 resources
Recipe: jackfruit::default
- Create image jackfruit-demo from machine jackfruit-demo with options {}[2015-08-19T15:38:43-07:00] INFO: [AWS EC2 200 0.154614 0 retries] describe_tags(:filters=>[{:name=>"</span>resource-id<span class="s2">",:values=>
- applying tags {"</span>From-Instance<span class="s2">"=>"</span>i-8b188520<span class="s2">"}[2015-08-19T15:38:44-07:00] INFO: Processing chef_data_bag[machine_image] action create (basic_chef_client::block line 62)
- create data bag machine_image at chefzero://localhost:8889[2015-08-19T15:38:44-07:00] INFO: Processing chef_data_bag_item[machine_image/jackfruit-demo] action create (basic_chef_client::block li
- create data bag item jackfruit-demo at chefzero://localhost:8889
- Image jackfruit-demo is now ready[2015-08-19T15:39:49-07:00] INFO: Processing aws_instance[jackfruit-demo] (i-8b188520) action destroy (basic_chef_client::block line 506)
- delete instance aws_instance[jackfruit-demo] (i-8b188520) in VPC vpc-e42b2f81 in us-east-1[2015-08-19T15:39:50-07:00] INFO: [AWS EC2 200 0.212792 0 retries] describe_instances(:instance_ids=>["</span>i
- delete node jackfruit-demo at chefzero://localhost:8889[2015-08-19T15:40:01-07:00] INFO: Processing chef_client[jackfruit-demo] action delete <span class="o">(</span>basic_chef_client::block line 44<span class="o">)</span>
- delete client jackfruit-demo at clients[2015-08-19T15:40:01-07:00] INFO: Processing chef_node[jackfruit-demo] action delete <span class="o">(</span>basic_chef_client::block line 91<span class="o">)</span>
<span class="k">*</span> ruby_block[share jackfruit-demo with the AWS Marketplace account] action run[2015-08-19T15:40:01-07:00] INFO: Processing ruby_block[share jackfruit-demo with the AWS Marketplace account] action ru
- execute the ruby block share jackfruit-demo with the AWS Marketplace account
<span class="o">[</span>2015-08-19T15:40:02-07:00] WARN: Skipping final node save because override_runlist was given
<span class="o">[</span>2015-08-19T15:40:02-07:00] INFO: Chef Run <span class="nb">complete </span><span class="k">in </span>594.834383 seconds
<span class="o">[</span>2015-08-19T15:40:02-07:00] INFO: Skipping removal of unused files from the cache
</code></pre>
</div>
</li>
</ul>
<p>595 seconds to create, converge, audit, publish and share your AMI in the AWS Marketplace.
Not bad for 10 minutes of work.</p>
<h3 id="run-the-security-scanner">Run the security scanner</h3>
<p>Login to the <a href="https://aws.amazon.com/marketplace/management/manage-products/?#/manage-amis.shared">AWS Marketplace Management UI</a> to locate your
new image.</p>
<p><img src="/images/aws_marketplace.png" align="midddle" /></p>
<p>Now you can intiate the AWS security scanning. Eventually I hope to have this
sIndextep automated but there’s not a public API and your Ec2/IAM credentials
cannot be used for authentication with the Marketplace Management console.</p>
<h3 id="going-further">Going further</h3>
<p>In this example we used our workstation node to run the publishing recipe. Ideally
publishing would be done as part of an automated CI/CD pipeline job that you can
trigger.</p>
<p>We use this same process to publish our software into the marketplace. You can find
our publishing recipes in our <a href="https://github.com/chef-partners/marketplace_image">marketplace_image cookbook</a></p>
<p>If you’d like to contribute, report bugs, or find more information, please visit
the <a href="https://github.com/chef-partners/marketplace_ami">marketplace_ami github repo</a></p>
Fixing the strange chef.io SSL errors when using Chef Provisioning2015-05-08T00:00:00+00:00https://blog.ryan.ec/2015/05/08/chef-provisioning-open-ssl-error<p>TLDR: Install the <a href="https://downloads.chef.io/chef-dk">ChefDK</a> for great good.</p>
<p>Today I was scheduled to work with another developer on Chef Provisioning on AWS
so I decided to dust off a few Chef Provisioning recipes I had lying around from
a demo a few months back. When I tried to provision one of them I was greeted with
a rather alarming problem</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code>ERROR: SSL Validation failure connecting to host: www.chef.io - SSL_connect <span class="nv">returned</span><span class="o">=</span>1 <span class="nv">errno</span><span class="o">=</span>0 <span class="nv">state</span><span class="o">=</span>SSLv3 <span class="nb">read </span>server certificate B: certificate verify failed
</code></pre>
</div>
<p>Interesting. Why are we contacting chef.io and why can’t I validate the SSL
certificate? After running <code class="highlighter-rouge">chef-client</code> with debug logging I determined that
Chef Provisioning was attempting to download and cache the chef-client omnibus
package for the machines that I was provisioning. The second issue is a much
bigger problem, but before we go any further I have to mention that this wouldn’t
have been a problem at all if my <code class="highlighter-rouge">chef-client</code> had been installed using the
<a href="https://downloads.chef.io/chef-dk">ChefDK</a>.
As as Ruby developer, I find myself using chruby to bounce between versions and
so I was using Chef the old-school manual gem install way. Big mistake.</p>
<p>Rather than switch to the ChefDK and get back to productive work, I decided to
hunt down the root cause. After verifying that the latest OpenSSL version was
installed and linked, my system gems were updated, and that I’d installed the
latest security bundle from Apple, I figured it must be a ca-cacert issue.</p>
<p>First I determined which cert Ruby was using</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> ruby -r openssl -e <span class="s2">"p OpenSSL::X509::DEFAULT_CERT_FILE"</span>
<span class="s2">"/usr/local/etc/openssl/cert.pem"</span>
</code></pre>
</div>
<p>Then updated updated it to reflect Apple’s latest certs</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> security find-certificate -a -p /Library/Keychains/System.keychain > /usr/local/etc/openssl/cert.pem
<span class="nv">$:</span> security find-certificate -a -p /System/Library/Keychains/SystemRootCertificates.keychain >> /usr/local/etc/openssl/cert.pem
</code></pre>
</div>
<p>Then tried running the provisioner again</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> chef-client -z provisioner.rb
...
ERROR: SSL Validation failure connecting to host: www.chef.io - SSL_connect <span class="nv">returned</span><span class="o">=</span>1 <span class="nv">errno</span><span class="o">=</span>0 <span class="nv">state</span><span class="o">=</span>SSLv3 <span class="nb">read </span>server certificate B: certificate verify failed
</code></pre>
</div>
<p>Damn. Okay, maybe Apple’s certs are no good. Let’s try the latest bundle from the
curl/Mozilla folks.</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> curl http://curl.haxx.se/ca/cacert.pem -o /usr/local/etc/openssl/cert.pem
<span class="nv">$:</span> chef-client -z provisioner.rb
...
ERROR: SSL Validation failure connecting to host: opscode-omnibus-packages.s3.amazonaws.com - SSL_connect <span class="nv">returned</span><span class="o">=</span>1 <span class="nv">errno</span><span class="o">=</span>0 <span class="nv">state</span><span class="o">=</span>SSLv3 <span class="nb">read </span>server certificate B: certificat
e verify failed
</code></pre>
</div>
<p>Sweet, certificate validation worked on chef.io but now we’re unable to verify the
S3 cert. We’re stuck in limbo here because Apple’s bundle doesn’t like
chef.io/fastly’s cert and curl’s bundle doesn’t like S3. After trying to verify
the S3 cert I stumbled upon a <a href="https://forums.aws.amazon.com/thread.jspa?threadID=164095">forum post</a> by Lamont that details that
support for 1024 bit signing keys has been phased out. Yikes. Until AWS reissues
their cert with a proper root signature we either have to abandon S3 or use an
older certificate bundle, the latter of which is how we’ve solved that problem
in the ChefDK.</p>
<p>I snagged a <a href="https://s3.amazonaws.com/uploads.hipchat.com/7557/2027616/rPfbosc2b4pPNh8/cacert.pem">copy of the certificate bundle from last August</a> (before
1024 bit keys were phased out) and gave it a go</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="nv">$:</span> curl https://s3.amazonaws.com/uploads.hipchat.com/7557/2027616/rPfbosc2b4pPNh8/cacert.pem -o /usr/local/etc/openssl/cert.pem
<span class="nv">$:</span> chef-client -z provisioner.rb
...
- create new file /Users/ryan/.chef/package_cache/chef_12.3.0-1_amd64.deb
...
</code></pre>
</div>
<p>Viola! Using the old bundle obviously isn’t ideal, but it unblocked me this time.</p>
Stubbing library class methods in ChefSpec2015-03-18T00:00:00+00:00https://blog.ryan.ec/2015/03/18/stubbing_class_methods_in_chefspec<p>One of the cool things about using Chef to automate your infrastructure is that
all of your infrastructure is represented as code. An awesome benefit of having
that representation is the ability to make assertions and test expected outcomes
in by unit testing. <a href="https://github.com/sethvargo/chefspec">ChefSpec</a>, a unit testing library built on top of
Rspec, comes bundled in the <a href="https://downloads.chef.io/chef-dk/">ChefDK</a> and allows you to create unit tests for
cookbooks. Unit testing cookbooks allows you to quickly iterate during development
because your assertions are usually limited to verifying that the right resources
passed the right messages and took the proper actions.</p>
<p>In ChefSpec all of the chef-runs are all in memory and the actions are no-ops.
Execution time is very quick so the feedback loop is much better than waiting for
an integration test to fully run.</p>
<p>Often times when unit testing you’ll find yourself stubbing various parts of your
program to isolate the pieces that you’re actually trying to test. ChefSpec makes
it very easy to stub node attributes but stubbing class methods in library
Classes and Modules gets a bit tricky. Let’s take a look at what I mean with an
example.</p>
<p>Here we have a cookbook called <code class="highlighter-rouge">demo</code> that has a default recipe and a library
that implements the Demo module. In the <code class="highlighter-rouge">Demo</code> module we’re going to have a method
<code class="highlighter-rouge">foo()</code> that will return a string <code class="highlighter-rouge">bar</code>. The helper methods in your cookbooks are
probably going to be doing something more complicated but we’ll run with this
example for simplicity.</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># recipes/default.rb</span>
<span class="n">log</span> <span class="no">Demo</span><span class="p">.</span><span class="nf">foo</span>
</code></pre>
</div>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># libraries/demo.rb</span>
<span class="k">module</span> <span class="nn">Demo</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">foo</span>
<span class="s1">'bar'</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
</div>
<p>Now write a ChefSpec test and attemp to stub <code class="highlighter-rouge">Demo.foo</code></p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># spec/spec_helper.rb</span>
<span class="nb">require_relative</span> <span class="s1">'../libraries/demo'</span>
<span class="nb">require</span> <span class="s1">'chefspec'</span>
<span class="nb">require</span> <span class="s1">'chefspec/berkshelf'</span>
<span class="n">describe</span> <span class="s1">'demo::default'</span> <span class="k">do</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:chef_run</span><span class="p">)</span> <span class="p">{</span> <span class="no">ChefSpec</span><span class="o">::</span><span class="no">SoloRunner</span><span class="p">.</span><span class="nf">converge</span><span class="p">(</span><span class="n">described_recipe</span><span class="p">)</span> <span class="p">}</span>
<span class="n">before</span> <span class="p">{</span> <span class="n">allow</span><span class="p">(</span><span class="no">Demo</span><span class="p">).</span><span class="nf">to</span> <span class="n">receive</span><span class="p">(</span><span class="ss">:foo</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="s1">'override'</span><span class="p">)</span> <span class="p">}</span>
<span class="n">it</span> <span class="s1">'stubs the demo library'</span> <span class="k">do</span>
<span class="n">expect</span><span class="p">(</span><span class="n">chef_run</span><span class="p">).</span><span class="nf">to</span> <span class="n">write_log</span><span class="p">(</span><span class="s1">'override'</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
</div>
<p>And run the test</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="gp">ryan@/tmp/demo(master):> </span>bundle <span class="nb">exec </span>rspec spec/spec_helper.rb
F
Failures:
1<span class="o">)</span> demo::default stubs the demo library
Failure/Error: expect<span class="o">(</span>chef_run<span class="o">)</span>.to write_log<span class="o">(</span><span class="s1">'override'</span><span class="o">)</span>
expected <span class="s2">"log[override]"</span> with action :write to be <span class="k">in </span>Chef run. Other log resources:
log[bar]
<span class="c"># ./spec/spec_helper.rb:11:in `block (2 levels) in <top (required)>'</span>
Finished <span class="k">in </span>0.09651 seconds <span class="o">(</span>files took 1.5 seconds to load<span class="o">)</span>
1 example, 1 failure
</code></pre>
</div>
<p>Ut oh, what’s going on here? The answer lies deep in the heart of the <code class="highlighter-rouge">chef-client</code>.
When we call <code class="highlighter-rouge">.converge</code> on our ChefSpec runner we actually run through the entire
chef-client compile and execute phases. During that setup we have to compile our
cookbook which requires us to <a href="https://github.com/chef/chef/blob/master/lib/chef/run_context/cookbook_compiler.rb#L187-198">load any library files</a> in the cookbook.</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># lib/chef/run_context/cookbook_compiler.rb</span>
<span class="k">def</span> <span class="nf">load_libraries_from_cookbook</span><span class="p">(</span><span class="n">cookbook_name</span><span class="p">)</span>
<span class="n">files_in_cookbook_by_segment</span><span class="p">(</span><span class="n">cookbook_name</span><span class="p">,</span> <span class="ss">:libraries</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="k">begin</span>
<span class="no">Chef</span><span class="o">::</span><span class="no">Log</span><span class="p">.</span><span class="nf">debug</span><span class="p">(</span><span class="s2">"Loading cookbook </span><span class="si">#{</span><span class="n">cookbook_name</span><span class="si">}</span><span class="s2">'s library file: </span><span class="si">#{</span><span class="n">filename</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">load</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>
<span class="vi">@events</span><span class="p">.</span><span class="nf">library_file_loaded</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>
<span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=></span> <span class="n">e</span>
<span class="vi">@events</span><span class="p">.</span><span class="nf">library_file_load_failed</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
<span class="k">raise</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
</div>
<p>You’ll see there on line #6 that we call <code class="highlighter-rouge">Kernel.load</code> on each library file in the
cookbook. Doing this will completely wipe out the stubs that we created on line #9
of our ChefSpec test. So how do we fix this?</p>
<p>If your library is only going to be used during converge time you can stub it by passing
a block to <code class="highlighter-rouge">converge()</code> with the class stubs</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># spec/spec_helper.rb</span>
<span class="n">let</span><span class="p">(</span><span class="ss">:chef_run</span><span class="p">)</span> <span class="k">do</span>
<span class="no">ChefSpec</span><span class="o">::</span><span class="no">SoloRunner</span><span class="p">.</span><span class="nf">converge</span><span class="p">(</span><span class="n">described_recipe</span><span class="p">)</span> <span class="k">do</span>
<span class="n">allow</span><span class="p">(</span><span class="no">Demo</span><span class="p">).</span><span class="nf">to</span> <span class="n">receive</span><span class="p">(</span><span class="ss">:foo</span><span class="p">).</span><span class="nf">and_return</span><span class="p">(</span><span class="s1">'override'</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="n">it</span> <span class="s1">'stubs the demo library'</span> <span class="k">do</span>
<span class="n">expect</span><span class="p">(</span><span class="n">chef_run</span><span class="p">).</span><span class="nf">to</span> <span class="n">write_log</span><span class="p">(</span><span class="s1">'override'</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
</div>
<p>That option won’t work in our case because the library is being used at compile
time for the name of the log resource.</p>
<p>You could monkey patch the chef-client in a library and omit loading the library</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># libraries/monkey_patches.rb</span>
<span class="k">def</span> <span class="nf">load_libraries_from_cookbook</span><span class="p">(</span><span class="n">cookbook_name</span><span class="p">)</span>
<span class="n">files_in_cookbook_by_segment</span><span class="p">(</span><span class="n">cookbook_name</span><span class="p">,</span> <span class="ss">:libraries</span><span class="p">).</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">filename</span><span class="o">|</span>
<span class="k">begin</span>
<span class="no">Chef</span><span class="o">::</span><span class="no">Log</span><span class="p">.</span><span class="nf">debug</span><span class="p">(</span><span class="s2">"Loading cookbook </span><span class="si">#{</span><span class="n">cookbook_name</span><span class="si">}</span><span class="s2">'s library file: </span><span class="si">#{</span><span class="n">filename</span><span class="si">}</span><span class="s2">"</span><span class="p">)</span>
<span class="no">Kernel</span><span class="p">.</span><span class="nf">load</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span>
<span class="vi">@events</span><span class="p">.</span><span class="nf">library_file_loaded</span><span class="p">(</span><span class="n">filename</span><span class="p">)</span> <span class="k">unless</span> <span class="n">cookbook_name</span> <span class="o">==</span> <span class="s1">'demo'</span>
<span class="k">rescue</span> <span class="no">Exception</span> <span class="o">=></span> <span class="n">e</span>
<span class="vi">@events</span><span class="p">.</span><span class="nf">library_file_load_failed</span><span class="p">(</span><span class="n">filename</span><span class="p">,</span> <span class="n">e</span><span class="p">)</span>
<span class="k">raise</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre>
</div>
<p>That would work but it’s a nasty hack at best. The cleanest option I’ve found is to
write your library in a way that it won’t be reloaded.</p>
<div class="language-ruby highlighter-rouge"><pre class="highlight"><code><span class="c1"># libraries/demo.rb</span>
<span class="k">module</span> <span class="nn">Demo</span>
<span class="k">def</span> <span class="nc">self</span><span class="o">.</span><span class="nf">foo</span>
<span class="s1">'bar'</span>
<span class="k">end</span>
<span class="k">end</span> <span class="k">unless</span> <span class="n">defined?</span><span class="p">(</span><span class="no">Demo</span><span class="p">)</span>
</code></pre>
</div>
<p>When the cookbook compiler tries to load the Demo library again it won’t
because we’ve previously loaded it in our <code class="highlighter-rouge">spec_helper.rb</code></p>
<p>If we run our tests again with the modified library we should see it pass!</p>
<div class="language-bash highlighter-rouge"><pre class="highlight"><code><span class="gp">ryan@/tmp/demo(master):> </span>bundle <span class="nb">exec </span>rspec spec/spec_helper.rb
.
Finished <span class="k">in </span>0.10684 seconds <span class="o">(</span>files took 1.58 seconds to load<span class="o">)</span>
1 example, 0 failures
</code></pre>
</div>