<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title><![CDATA[0xpebbles.org]]></title>
    <link><![CDATA[http://blog.0xpebbles.org]]></link>
    <description><![CDATA[0xpebbles blog]]></description>
    <lastBuildDate>Fri, 09 Apr 2021 08:37:23 +0200</lastBuildDate>
    <pubDate>Fri, 09 Apr 2021 08:37:23 +0200</pubDate>
    <language>en</language>

<!-- 20150319 -->    <item>
      <title>shell script for S3-upload via curl using AWS version 4 signatures</title>
      <link>http://blog.0xpebbles.org/shell-script-for-S3-upload-via-curl-using-AWS-version-4-signatures</link>
      <pubDate>19 Mar 2015 00:00:00 +0000</pubDate>
      <content:encoded><![CDATA[

  

<p>
This is a more modern version of <a href="/shell-script-to-upload-to-S3-via-curl">this script</a>, switching
to AWS version 4 signatures that are mandatory for AWS regions created after January 2014. It also works with
older regions as they seem to support the new signature format, as well.
</p>

<p>
The script's interface is a bit easier and more intuitive, too, and allows setting the access permissions, now. See the script's
own help-text for information and examples on how to use it.
</p>

<style type="text/css">
<!--
pre.sh { font-family: monospace; color: #dadada; background-color: #000000; }
.Comment { color: #626262; }
.Constant { color: #87afff; font-weight: bold; }
.Special { color: #00d700; font-weight: bold; }
.Identifier { color: #ffd700; }
.Statement { color: #ff8700; font-weight: bold; }
.PreProc { color: #008787; }
.String { color: #8787ff; }
.shShellVariables { color: #87afff; }
-->
</style>
<pre class="sh">
<span class="Comment">#!/bin/sh</span>

<span class="Identifier">usage()</span>
<span class="Identifier">{</span>
    <span class="Statement">cat</span> <span class="Statement">&lt;&lt;USAGE</span>

<span class="String">Simple script uploading a file to S3. Supports AWS signature version 4, custom</span>
<span class="String">region, permissions and mime-types. Uses Content-MD5 header to guarantee</span>
<span class="String">uncorrupted file transfer.</span>

<span class="String">Usage:</span>
<span class="String">  </span><span class="Special">`basename </span><span class="shShellVariables">$0</span><span class="Special">`</span><span class="String"> aws_ak aws_sk bucket srcfile targfile [acl] [mime_type]</span>

<span class="String">Where &lt;arg&gt; is one of:</span>
<span class="String">  aws_ak     access key ('' for upload to public writable bucket)</span>
<span class="String">  aws_sk     secret key ('' for upload to public writable bucket)</span>
<span class="String">  bucket     bucket name (with optional @region suffix, default is us-east-1)</span>
<span class="String">  srcfile    path to source file</span>
<span class="String">  targfile   path to target (dir if it ends with '/', relative to bucket root)</span>
<span class="String">  acl        s3 access permissions (default: public-read)</span>
<span class="String">  mime_type  optional mime-type (tries to guess if omitted)</span>

<span class="String">Dependencies:</span>
<span class="String">  To run, this shell script depends on command-line curl and openssl, as well</span>
<span class="String">  as standard Unix tools</span>

<span class="String">Examples:</span>
<span class="String">  To upload file '~/blog/media/image.png' to bucket 'storage' in region</span>
<span class="String">  'eu-central-1' with key (path relative to bucket) 'media/image.png':</span>

<span class="String">    </span><span class="Special">`basename </span><span class="shShellVariables">$0</span><span class="Special">`</span><span class="String"> ACCESS SECRET storage@eu-central-1 </span><span class="Special">\\</span>
<span class="String">      ~/blog/image.png media/</span>

<span class="String">  To upload file '~/blog/media/image.png' to public-writable bucket 'storage'</span>
<span class="String">  in default region 'us-east-1' with key (path relative to bucket) 'x/y.png':</span>

<span class="String">    </span><span class="Special">`basename </span><span class="shShellVariables">$0</span><span class="Special">`</span><span class="String"> '' '' storage ~/blog/image.png x/y.png</span>

<span class="Statement">USAGE</span>
    <span class="Statement">exit</span> <span class="Constant">0</span>
<span class="Identifier">}</span>

<span class="Identifier">guessmime()</span>
<span class="Identifier">{</span>
    <span class="Identifier">mime</span>=<span class="Special">`file </span><span class="Special">-b</span><span class="Special"> </span><span class="Special">--mime-type</span><span class="Special"> </span><span class="shShellVariables">$1</span><span class="Special">`</span>
    <span class="Statement">if </span><span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$mime</span><span class="Statement">&quot;</span> <span class="Statement">=</span> <span class="String">&quot;text/plain&quot;</span> <span class="Statement">]</span>; <span class="Statement">then</span>
        <span class="Statement">case</span> <span class="shShellVariables">$1</span> <span class="Statement">in</span>
            *.css<span class="Statement">)</span>           <span class="Identifier">mime</span>=text/css<span class="Statement">;;</span>
            *.ttf<span class="Statement">|</span>*.otf<span class="Statement">)</span>     <span class="Identifier">mime</span>=application/font-sfnt<span class="Statement">;;</span>
            *.woff<span class="Statement">)</span>          <span class="Identifier">mime</span>=application/font-woff<span class="Statement">;;</span>
            *.woff2<span class="Statement">)</span>         <span class="Identifier">mime</span>=font/woff<span class="Constant">2</span><span class="Statement">;;</span>
            *rss*.xml<span class="Statement">|</span>*.rss<span class="Statement">)</span> <span class="Identifier">mime</span>=application/rss+xml<span class="Statement">;;</span>
            *<span class="Statement">)</span>               <span class="Statement">if </span>head <span class="shShellVariables">$1</span> | <span class="Statement">grep</span> <span class="Statement">'</span><span class="String">&lt;html.*&gt;</span><span class="Statement">'</span> <span class="Statement">&gt;</span>/dev/null; <span class="Statement">then</span> <span class="Identifier">mime</span>=text/html; <span class="Statement">fi</span><span class="Statement">;;</span>
        <span class="Statement">esac</span>
    <span class="Statement">fi</span>
    <span class="Statement">printf</span> <span class="Statement">&quot;</span><span class="shShellVariables">$mime</span><span class="Statement">&quot;</span>
<span class="Identifier">}</span>

<span class="Statement">if </span><span class="Statement">[</span> <span class="shShellVariables">$#</span> <span class="Statement">-lt</span> <span class="Constant">5</span> <span class="Statement">]</span>; <span class="Statement">then</span> usage; <span class="Statement">fi</span>

<span class="Comment"># Inputs.</span>
<span class="Identifier">aws_ak</span>=<span class="Statement">&quot;</span><span class="shShellVariables">$1</span><span class="Statement">&quot;</span>                                                              <span class="Comment"># access key</span>
<span class="Identifier">aws_sk</span>=<span class="Statement">&quot;</span><span class="shShellVariables">$2</span><span class="Statement">&quot;</span>                                                              <span class="Comment"># secret key</span>
<span class="Identifier">bucket</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="shShellVariables">$3</span><span class="Special"> | awk </span><span class="Statement">'</span><span class="String">BEGIN{FS=&quot;@&quot;}{print $1}</span><span class="Statement">'</span><span class="Special">`</span>                       <span class="Comment"># bucket name</span>
<span class="Identifier">region</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="shShellVariables">$3</span><span class="Special"> | awk </span><span class="Statement">'</span><span class="String">BEGIN{FS=&quot;@&quot;}{print ($2==&quot;&quot;?&quot;us-east-1&quot;:$2)}</span><span class="Statement">'</span><span class="Special">`</span>  <span class="Comment"># region name</span>
<span class="Identifier">srcfile</span>=<span class="Statement">&quot;</span><span class="shShellVariables">$4</span><span class="Statement">&quot;</span>                                                             <span class="Comment"># source file</span>
<span class="Identifier">targfile</span>=<span class="Special">`</span><span class="Statement">echo</span><span class="String"> -n </span><span class="Statement">&quot;</span><span class="shShellVariables">$5</span><span class="Statement">&quot;</span><span class="String"> </span><span class="Statement">|</span><span class="Special"> </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="String">s/</span><span class="Special">\/</span><span class="String">$/</span><span class="Special">\/</span><span class="shShellVariables">$(</span><span class="Special">basename </span><span class="shShellVariables">$srcfile</span><span class="shShellVariables">)</span><span class="String">/</span><span class="Statement">&quot;</span><span class="Special">`</span>            <span class="Comment"># target file</span>
<span class="Identifier">acl</span>=<span class="PreProc">${</span><span class="shShellVariables">6</span><span class="Statement">:-</span><span class="Statement">'</span><span class="String">public-read</span><span class="Statement">'</span><span class="PreProc">}</span>                                                  <span class="Comment"># s3 perms</span>
<span class="Identifier">mime</span>=<span class="PreProc">${</span><span class="shShellVariables">7</span><span class="Statement">:-</span><span class="Statement">&quot;</span><span class="Special">`guessmime </span><span class="Statement">&quot;</span><span class="shShellVariables">$srcfile</span><span class="Statement">&quot;</span><span class="Special">`</span><span class="Statement">&quot;</span><span class="PreProc">}</span>                                      <span class="Comment"># mime type</span>
<span class="Identifier">md5</span>=<span class="Special">`openssl md5 </span><span class="Special">-binary</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$srcfile</span><span class="Statement">&quot;</span><span class="Special"> | openssl base64`</span>


<span class="Comment"># Create signature if not public upload.</span>
<span class="Identifier">key_and_sig_args</span>=<span class="Statement">''</span>
<span class="Statement">if </span><span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$aws_ak</span><span class="Statement">&quot;</span> <span class="Statement">!=</span> <span class="Statement">&quot;&quot;</span> <span class="Statement">]</span> &amp;&amp; <span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$aws_sk</span><span class="Statement">&quot;</span> <span class="Statement">!=</span> <span class="Statement">&quot;&quot;</span> <span class="Statement">]</span>; <span class="Statement">then</span>

    <span class="Comment"># Need current and file upload expiration date. Handle GNU and BSD date command style to get tomorrow's date.</span>
    <span class="Identifier">date</span>=<span class="Special">`date </span><span class="Special">-u</span><span class="Special"> +%Y%m%dT%H%M%SZ`</span>
    <span class="Identifier">expdate</span>=<span class="Special">`if ! date </span><span class="Special">-v</span><span class="Special">+1d +%Y-%m-%d </span><span class="Constant">2</span><span class="Special">&gt;/dev/null; then date </span><span class="Special">-d</span><span class="Special"> tomorrow +%Y-%m-%d; fi`</span>
    <span class="Identifier">expdate_s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="shShellVariables">$expdate</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> s/-//g`</span> <span class="Comment"># without dashes, as we need both formats below</span>
    <span class="Identifier">service</span>=<span class="Statement">'</span><span class="String">s3</span><span class="Statement">'</span>

    <span class="Comment"># Generate policy and sign with secret key following AWS Signature version 4, below</span>
    <span class="Identifier">p</span>=<span class="shShellVariables">$(</span><span class="Statement">cat</span><span class="Special"> &lt;&lt;POLICY | openssl base64</span>
<span class="Special">{ </span><span class="Statement">&quot;</span><span class="String">expiration</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="PreProc">${</span><span class="shShellVariables">expdate</span><span class="PreProc">}</span><span class="String">T12:00:00.000Z</span><span class="Statement">&quot;</span><span class="Special">,</span>
<span class="Special">  </span><span class="Statement">&quot;</span><span class="String">conditions</span><span class="Statement">&quot;</span><span class="Special">: [</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">acl</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$acl</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">bucket</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$bucket</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">starts-with</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;</span><span class="Special">\$</span><span class="String">key</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;&quot;</span><span class="Special">],</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">starts-with</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;</span><span class="Special">\$</span><span class="String">content-type</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;&quot;</span><span class="Special">],</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">content-length-range</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Constant">1</span><span class="Special">, `</span><span class="Statement">ls</span><span class="Special"> </span><span class="Special">-l</span><span class="Special"> </span><span class="Special">-H</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$srcfile</span><span class="Statement">&quot;</span><span class="Special"> | awk </span><span class="Statement">'</span><span class="String">{print $5}</span><span class="Statement">'</span><span class="Special"> | head </span><span class="Constant">-1</span><span class="Special">`],</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">content-md5</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$md5</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">x-amz-date</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$date</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">x-amz-credential</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$aws_ak</span><span class="String">/</span><span class="shShellVariables">$expdate_s</span><span class="String">/</span><span class="shShellVariables">$region</span><span class="String">/</span><span class="shShellVariables">$service</span><span class="String">/aws4_request</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">x-amz-algorithm</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="String">AWS4-HMAC-SHA256</span><span class="Statement">&quot;</span><span class="Special"> }</span>
<span class="Special">  ]</span>
<span class="Special">}</span>
<span class="Special">POLICY</span>
<span class="Special">    </span><span class="shShellVariables">)</span>

    <span class="Comment"># AWS4-HMAC-SHA256 signature</span>
    <span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$expdate_s</span><span class="Statement">&quot;</span><span class="Special">   | openssl sha256 </span><span class="Special">-hmac</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="String">AWS4</span><span class="shShellVariables">$aws_sk</span><span class="Statement">&quot;</span><span class="Special">           </span><span class="Special">-hex</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">'</span><span class="String">s/(stdin)= //</span><span class="Statement">'</span><span class="Special">`</span>
    <span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$region</span><span class="Statement">&quot;</span><span class="Special">      | openssl sha256 </span><span class="Special">-mac</span><span class="Special"> HMAC </span><span class="Special">-macopt</span><span class="Special"> hexkey:</span><span class="Statement">&quot;</span><span class="shShellVariables">$s</span><span class="Statement">&quot;</span><span class="Special"> </span><span class="Special">-hex</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">'</span><span class="String">s/(stdin)= //</span><span class="Statement">'</span><span class="Special">`</span>
    <span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$service</span><span class="Statement">&quot;</span><span class="Special">     | openssl sha256 </span><span class="Special">-mac</span><span class="Special"> HMAC </span><span class="Special">-macopt</span><span class="Special"> hexkey:</span><span class="Statement">&quot;</span><span class="shShellVariables">$s</span><span class="Statement">&quot;</span><span class="Special"> </span><span class="Special">-hex</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">'</span><span class="String">s/(stdin)= //</span><span class="Statement">'</span><span class="Special">`</span>
    <span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="String">aws4_request</span><span class="Statement">&quot;</span><span class="Special"> | openssl sha256 </span><span class="Special">-mac</span><span class="Special"> HMAC </span><span class="Special">-macopt</span><span class="Special"> hexkey:</span><span class="Statement">&quot;</span><span class="shShellVariables">$s</span><span class="Statement">&quot;</span><span class="Special"> </span><span class="Special">-hex</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">'</span><span class="String">s/(stdin)= //</span><span class="Statement">'</span><span class="Special">`</span>
    <span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$p</span><span class="Statement">&quot;</span><span class="Special">           | openssl sha256 </span><span class="Special">-mac</span><span class="Special"> HMAC </span><span class="Special">-macopt</span><span class="Special"> hexkey:</span><span class="Statement">&quot;</span><span class="shShellVariables">$s</span><span class="Statement">&quot;</span><span class="Special"> </span><span class="Special">-hex</span><span class="Special"> | </span><span class="Statement">sed</span><span class="Special"> </span><span class="Statement">'</span><span class="String">s/(stdin)= //</span><span class="Statement">'</span><span class="Special">`</span>

    <span class="Identifier">key_and_sig_args</span>=<span class="Statement">&quot;</span><span class="String">-F X-Amz-Credential=</span><span class="shShellVariables">$aws_ak</span><span class="String">/</span><span class="shShellVariables">$expdate_s</span><span class="String">/</span><span class="shShellVariables">$region</span><span class="String">/</span><span class="shShellVariables">$service</span><span class="String">/aws4_request -F X-Amz-Algorithm=AWS4-HMAC-SHA256 -F X-Amz-Signature=</span><span class="shShellVariables">$s</span><span class="String"> -F X-Amz-Date=</span><span class="PreProc">${</span><span class="shShellVariables">date</span><span class="PreProc">}</span><span class="Statement">&quot;</span>
<span class="Statement">fi</span>


<span class="Comment"># Upload. Supports anonymous upload if bucket is public-writable, and keys are set to ''.</span>
<span class="Statement">echo</span><span class="String"> </span><span class="Statement">&quot;</span><span class="String">Uploading: </span><span class="shShellVariables">$srcfile</span><span class="String"> (</span><span class="shShellVariables">$mime</span><span class="String">) to </span><span class="shShellVariables">$bucket</span><span class="String">:</span><span class="shShellVariables">$targfile</span><span class="Statement">&quot;</span>
curl                            <span class="Statement">\</span>
    <span class="Special">-#</span> <span class="Special">-k</span>                       <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Identifier">key</span>=<span class="shShellVariables">$targfile</span>            <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Identifier">acl</span>=<span class="shShellVariables">$acl</span>                 <span class="Statement">\</span>
    <span class="shShellVariables">$key_and_sig_args</span>           <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">Policy=</span><span class="shShellVariables">$p</span><span class="Statement">&quot;</span>              <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">Content-MD5=</span><span class="shShellVariables">$md5</span><span class="Statement">&quot;</span>       <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">Content-Type=</span><span class="shShellVariables">$mime</span><span class="Statement">&quot;</span>     <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">file=@</span><span class="shShellVariables">$srcfile</span><span class="Statement">&quot;</span>         <span class="Statement">\</span>
    https://<span class="PreProc">${</span><span class="shShellVariables">bucket</span><span class="PreProc">}</span>.s3.amazonaws.com/ | <span class="Statement">cat</span> <span class="Comment"># pipe through cat so curl displays upload progress bar, *and* response</span>

</pre>

]]></content:encoded>
    </item>

<!-- 20140822 -->    <item>
      <title>shell script to upload to S3 via curl</title>
      <link>http://blog.0xpebbles.org/shell-script-to-upload-to-S3-via-curl</link>
      <pubDate>22 Aug 2014 00:00:00 +0000</pubDate>
      <content:encoded><![CDATA[

  

<p>
The following is a small shell script to upload files to S3. It was born out of the frustration with
other AWS toolkits either being bloated, having lots of dependencies, needing some scripting language
or having constantly changing interfaces across versions.
</p>

<p>
The script has exactly two dependencies other than standard Unix commands, namely <b>openssl</b> (to put
together a signature needed for upload) and <b>curl</b> to upload via HTTP POST. It also does a tiny little
bit of mime-type guessing.
</p>
<p>
<em><b>Update/Note:</b> This does not handle AWS version 4 signatures, however, check a more recent blog post
for a version that does.</em>
</p>

<style type="text/css">
<!--
pre.sh { font-family: monospace; color: #dadada; background-color: #000000; }
.Comment { color: #626262; }
.Constant { color: #87afff; font-weight: bold; }
.Special { color: #00d700; font-weight: bold; }
.Identifier { color: #ffd700; }
.Statement { color: #ff8700; font-weight: bold; }
.PreProc { color: #008787; }
.String { color: #8787ff; }
.shShellVariables { color: #87afff; }
-->
</style>
<pre class="sh">
<span class="Comment">#!/bin/sh</span>
<span class="Identifier">usage()</span>
<span class="Identifier">{</span>
	<span class="Statement">cat</span> <span class="Statement">&lt;&lt;USAGE</span>

<span class="String">Usage:</span>
<span class="String">  </span><span class="Special">`basename </span><span class="shShellVariables">$0</span><span class="Special">`</span><span class="String"> aws_ak aws_sk bucket srcpath file [mime_type]</span>

<span class="String">Where &lt;arg&gt; is one of:</span>
<span class="String">  aws_ak     access key ('' for upload to public writable bucket)</span>
<span class="String">  aws_sk     secret key ('' for upload to public writable bucket)</span>
<span class="String">  bucket     bucket name</span>
<span class="String">  srcpath    local path prefix to file (end with /, for wd use ./)</span>
<span class="String">  file       rest of path to file and path relative to bucket root</span>
<span class="String">  mime_type  optional mime-type (tries to guess if omitted)</span>

<span class="String">Dependencies:</span>
<span class="String">  To run, this shell script depends on command-line curl and openssl</span>

<span class="String">Example:</span>
<span class="String">  To upload file '~/blog/media/image.png' to bucket 'storage' with</span>
<span class="String">  key (path relative to bucket) 'media/image.png':</span>

<span class="String">    </span><span class="Special">`basename </span><span class="shShellVariables">$0</span><span class="Special">`</span><span class="String"> ACCESS SECRET storage ~/blog/ media/image.png</span>

<span class="Statement">USAGE</span>
	<span class="Statement">exit</span> <span class="Constant">0</span>
<span class="Identifier">}</span>

<span class="Statement">if </span><span class="Statement">[</span> <span class="shShellVariables">$#</span> <span class="Statement">-lt</span> <span class="Constant">5</span> <span class="Statement">]</span>; <span class="Statement">then</span> usage; <span class="Statement">fi</span>

<span class="Comment"># Inputs.</span>
<span class="Identifier">aws_ak</span>=<span class="shShellVariables">$1</span>  <span class="Comment"># access key</span>
<span class="Identifier">aws_sk</span>=<span class="shShellVariables">$2</span>  <span class="Comment"># secret key</span>
<span class="Identifier">bucket</span>=<span class="shShellVariables">$3</span>  <span class="Comment"># bucket name</span>
<span class="Identifier">srcpath</span>=<span class="shShellVariables">$4</span> <span class="Comment"># local path prefix to file</span>
<span class="Identifier">file</span>=<span class="shShellVariables">$5</span>    <span class="Comment"># rest of path to file and path relative to bucket root</span>
<span class="Statement">if </span><span class="Statement">[</span> <span class="shShellVariables">$#</span> <span class="Statement">-gt</span> <span class="Constant">5</span> <span class="Statement">]</span>; <span class="Statement">then</span>
	<span class="Identifier">mime</span>=<span class="shShellVariables">$6</span>;
<span class="Statement">else</span>
	<span class="Identifier">mime</span>=<span class="Special">`file </span><span class="Special">-b</span><span class="Special"> </span><span class="Special">--mime-type</span><span class="Special"> </span><span class="shShellVariables">$srcpath$file</span><span class="Special">`</span>
	<span class="Statement">if </span><span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$mime</span><span class="Statement">&quot;</span> <span class="Statement">=</span> <span class="String">&quot;text/plain&quot;</span> <span class="Statement">]</span>; <span class="Statement">then</span>
		<span class="Statement">case</span> <span class="shShellVariables">$file</span> <span class="Statement">in</span>
			*.css<span class="Statement">)</span> <span class="Identifier">mime</span>=text/css<span class="Statement">;;</span>
			*<span class="Statement">)</span>     <span class="Statement">if </span>head <span class="shShellVariables">$srcpath$file</span> | <span class="Statement">grep</span> <span class="Statement">'</span><span class="String">&lt;html&gt;</span><span class="Statement">'</span> <span class="Statement">&gt;</span>/dev/null; <span class="Statement">then</span> <span class="Identifier">mime</span>=text/html; <span class="Statement">fi</span><span class="Statement">;;</span>
		<span class="Statement">esac</span>
	<span class="Statement">fi</span>
<span class="Statement">fi</span>

<span class="Comment"># Generate policy and sign with secret key. Handle GNU and BSD date command style to get tomorrow's date.</span>
<span class="Identifier">p</span>=<span class="shShellVariables">$(</span><span class="Statement">cat</span><span class="Special"> &lt;&lt;POLICY | openssl base64</span>
<span class="Special">{ </span><span class="Statement">&quot;</span><span class="String">expiration</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="Special">`if ! date </span><span class="Special">-v</span><span class="Special">+1d +%Y-%m-%d </span><span class="Constant">2</span><span class="Special">&gt;/dev/null; then date </span><span class="Special">-d</span><span class="Special"> tomorrow +%Y-%m-%d; fi`</span><span class="String">T12:00:00.000Z</span><span class="Statement">&quot;</span><span class="Special">,</span>
<span class="Special">  </span><span class="Statement">&quot;</span><span class="String">conditions</span><span class="Statement">&quot;</span><span class="Special">: [</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">acl</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="String">public-read</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    {</span><span class="Statement">&quot;</span><span class="String">bucket</span><span class="Statement">&quot;</span><span class="Special">: </span><span class="Statement">&quot;</span><span class="shShellVariables">$bucket</span><span class="Statement">&quot;</span><span class="Special"> },</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">starts-with</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;</span><span class="Special">\$</span><span class="String">key</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;&quot;</span><span class="Special">],</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">starts-with</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;</span><span class="Special">\$</span><span class="String">content-type</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Statement">&quot;&quot;</span><span class="Special">],</span>
<span class="Special">    [</span><span class="Statement">&quot;</span><span class="String">content-length-range</span><span class="Statement">&quot;</span><span class="Special">, </span><span class="Constant">1</span><span class="Special">, </span><span class="Constant">20971520</span><span class="Special">]</span>
<span class="Special">  ]</span>
<span class="Special">}</span>
<span class="Special">POLICY</span>
<span class="shShellVariables">)</span>
<span class="Identifier">s</span>=<span class="Special">`</span><span class="Statement">printf</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$p</span><span class="Statement">&quot;</span><span class="Special"> | openssl sha1 </span><span class="Special">-hmac</span><span class="Special"> </span><span class="Statement">&quot;</span><span class="shShellVariables">$aws_sk</span><span class="Statement">&quot;</span><span class="Special"> </span><span class="Special">-binary</span><span class="Special"> | openssl base64`</span>

<span class="Comment"># Upload. Supports anonymous upload if bucket is public-writable, and keys are set to ''.</span>
<span class="Statement">echo</span><span class="String"> </span><span class="Statement">&quot;</span><span class="String">Uploading: </span><span class="shShellVariables">$file</span><span class="String"> (</span><span class="shShellVariables">$mime</span><span class="String">) to bucket </span><span class="shShellVariables">$bucket</span><span class="Statement">&quot;</span>
<span class="Identifier">key_and_sig_args</span>=<span class="Statement">''</span>
<span class="Statement">if </span><span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$aws_ak</span><span class="Statement">&quot;</span> <span class="Statement">!=</span> <span class="Statement">&quot;&quot;</span> <span class="Statement">]</span> &amp;&amp; <span class="Statement">[</span> <span class="Statement">&quot;</span><span class="shShellVariables">$aws_sk</span><span class="Statement">&quot;</span> <span class="Statement">!=</span> <span class="Statement">&quot;&quot;</span> <span class="Statement">]</span>; <span class="Statement">then</span>
    <span class="Identifier">key_and_sig_args</span>=<span class="Statement">&quot;</span><span class="String">-F AWSAccessKeyId=</span><span class="shShellVariables">$aws_ak</span><span class="String"> -F Signature=</span><span class="shShellVariables">$s</span><span class="Statement">&quot;</span>
<span class="Statement">fi</span>

curl                            <span class="Statement">\</span>
    <span class="Special">-#</span> <span class="Special">-k</span>                       <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Identifier">key</span>=<span class="shShellVariables">$file</span>                <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Identifier">acl</span>=public-<span class="Statement">read</span>          <span class="Statement">\</span>
    <span class="shShellVariables">$key_and_sig_args</span>           <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">Policy=</span><span class="shShellVariables">$p</span><span class="Statement">&quot;</span>              <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Statement">&quot;</span><span class="String">Content-Type=</span><span class="shShellVariables">$mime</span><span class="Statement">&quot;</span>     <span class="Statement">\</span>
    <span class="Special">-F</span> <span class="Identifier">file</span>=@<span class="shShellVariables">$srcpath$file</span>      <span class="Statement">\</span>
    https://<span class="PreProc">${</span><span class="shShellVariables">bucket</span><span class="PreProc">}</span>.s3.amazonaws.com/ | tee <span class="Comment"># pass response through tee so curl displays upload progress bar, *and* response</span>

</pre>

]]></content:encoded>
    </item>




  </channel>
</rss>

