<rss xmlns:a10="http://www.w3.org/2005/Atom" version="2.0"><channel><title /><link>http://www.philliphaydon.com/feed.xml</link><description /><item><guid isPermaLink="true">http://www.philliphaydon.com/2017/05/10/part5-configuring-the-vpc-so-we-can-call-the-database/</guid><link>http://www.philliphaydon.com/2017/05/10/part5-configuring-the-vpc-so-we-can-call-the-database/</link><a10:author><a10:name /></a10:author><category>aws</category><category>dotnetcore</category><category>lambda</category><category>marten</category><category>postgresql</category><category>vpc</category><title>Configuring the VPC so we can call the database</title><description>&lt;p&gt;Ahh the final part.&lt;/p&gt;

&lt;p&gt;So if you're using AWS you're most likely using RDS. If you're using RDS then you've got a database, and if it's in production then hopefully you've disabled public access and haven't configured it to allow a connection from any IP address.&lt;/p&gt;

&lt;p&gt;If this is the case then the moment we test our Lambda, it's going to fail with a timeout. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
  "errorMessage": "2017-03-23T17:12:36.269Z e0474e8b-0feb-11e7-a0fe-276de97e95bf Task timed out after 15.00 seconds"
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This happens because it cannot connect to the database, if the lambda execution time is less than your database timeout then this happens. Otherwise if the inverse is true you would get the database timeout exception rather than the lambda timeout exception.&lt;/p&gt;

</description><pubDate>Tue, 09 May 2017 16:00:00 Z</pubDate><a10:updated>2017-05-09T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Ahh the final part.&lt;/p&gt;

&lt;p&gt;So if you're using AWS you're most likely using RDS. If you're using RDS then you've got a database, and if it's in production then hopefully you've disabled public access and haven't configured it to allow a connection from any IP address.&lt;/p&gt;

&lt;p&gt;If this is the case then the moment we test our Lambda, it's going to fail with a timeout. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
  "errorMessage": "2017-03-23T17:12:36.269Z e0474e8b-0feb-11e7-a0fe-276de97e95bf Task timed out after 15.00 seconds"
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This happens because it cannot connect to the database, if the lambda execution time is less than your database timeout then this happens. Otherwise if the inverse is true you would get the database timeout exception rather than the lambda timeout exception.&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;p&gt;So RDS instances are created with a VPC by default, assuming that everything is created with default settings and the vpc is named rds-launch-wizard, we can begin configuring the Lambda to have permission to connect to RDS.&lt;/p&gt;

&lt;h1&gt;Create a role with VPC support&lt;/h1&gt;

&lt;p&gt;Let's start by opening up IAM and creating a new role. &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Select AWS Lambda from the Role Types.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;The trust step is skipped and it will go directly to Attach Policy.&lt;/p&gt;

&lt;p&gt;Under attach policy you want to add:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;AWS Lambda Basic Execution Role&lt;/li&gt;
&lt;li&gt;2 AWS Lambda VPC Access Execution Role&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-03.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;On the final step we can name our role, such as &lt;code&gt;lambda-with-vpc-access&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-04.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Hit create and now we should have a new role ready for use in our Lambda.&lt;/p&gt;

&lt;h1&gt;Upload / Configure&lt;/h1&gt;

&lt;p&gt;Let's start by uploading the lambda and configuring it. Let's create a new blank function like we did in part 2.&lt;/p&gt;

&lt;p&gt;Give it a name like 'CreateProduct', and select the package we created in part 4.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-05.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Set the Handler to:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;HelloWorldLambda::HelloWorldLambda.Handler::Save
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Remember, this correlates to the dll::namespace.class::function&lt;/p&gt;

&lt;p&gt;Under the Role, pick 'Choose an existing role', the role we created above should show:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-06.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now, expand out the Advanced settings section.&lt;/p&gt;

&lt;p&gt;You can modify the Memory / Timeout now or change it later. To test with I set mine to 512mb and 15 second timeout. Once it's all working I adjust it down.&lt;/p&gt;

&lt;p&gt;Under the VPC, select the VPC that your RDS instance uses, and add the subnet's its in, as well as the security group. &lt;/p&gt;

&lt;p&gt;For example, this is my RDS Configuration:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-07.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Vs the VPC Configuration of the Lambda:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-08.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Note: Yes I blanked things out because things and stuff don't want people figuring out my account info.&lt;/p&gt;

&lt;p&gt;Everything else you can leave default.&lt;/p&gt;

&lt;p&gt;Now review, and create that Lambda!&lt;/p&gt;

&lt;h1&gt;Testing&lt;/h1&gt;

&lt;p&gt;Now that it's saved, we can test our lambda, so let's click on Actions and pick 'Configure test event'&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-09.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now click Save and test, and unlike the start of this post, we should get successful execution result.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-10.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;When testing with Lambda targeting RDS, it usually has a startup time of 2-3 seconds, but every execution after that seems super fast. &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-5-11.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Enjoy. &lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2017/03/21/part4-building-and-publishing-a-dotnet-core-app-using-aws-tools/</guid><link>http://www.philliphaydon.com/2017/03/21/part4-building-and-publishing-a-dotnet-core-app-using-aws-tools/</link><a10:author><a10:name /></a10:author><category>aws</category><category>dotnetcore</category><category>lambda</category><category>marten</category><category>postgresql</category><title>Building / Publishing a .NET Core app using AWS Tools</title><description>&lt;p&gt;Now that we've got everything, we need to deploy it, however if we build it using dotnet clr it will fail. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;"cause": {
  "errorType": "TypeInitializationException",
  "errorMessage": "The type initializer for 'Npgsql.TypeHandlerRegistry' threw an exception.",
  "stackTrace": [
    "at Npgsql.TypeHandlerRegistry.Setup(NpgsqlConnector connector, NpgsqlTimeout timeout, Boolean async)",
    "at Npgsql.NpgsqlConnector.&amp;lt;Open&amp;gt;d__137.MoveNext()",
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;That's because Npgsql need's to load some dependencies but the Lambda's don't understand that. No matter how I publish, restore for a target runtime and publish agains that, it doesn't seem to work. However, AWS have their own toolkit for publishing lambda's. &lt;/p&gt;

</description><pubDate>Mon, 20 Mar 2017 16:00:00 Z</pubDate><a10:updated>2017-03-20T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Now that we've got everything, we need to deploy it, however if we build it using dotnet clr it will fail. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;"cause": {
  "errorType": "TypeInitializationException",
  "errorMessage": "The type initializer for 'Npgsql.TypeHandlerRegistry' threw an exception.",
  "stackTrace": [
    "at Npgsql.TypeHandlerRegistry.Setup(NpgsqlConnector connector, NpgsqlTimeout timeout, Boolean async)",
    "at Npgsql.NpgsqlConnector.&amp;lt;Open&amp;gt;d__137.MoveNext()",
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;That's because Npgsql need's to load some dependencies but the Lambda's don't understand that. No matter how I publish, restore for a target runtime and publish agains that, it doesn't seem to work. However, AWS have their own toolkit for publishing lambda's. &lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;h1&gt;Installing the toolkit so we can publish&lt;/h1&gt;

&lt;p&gt;Instead of using the dotnet clr to publish we can install the &lt;a href="https://www.nuget.org/packages/Amazon.Lambda.Tools"&gt;AWS Lambda Toolkit&lt;/a&gt;. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;dotnet add .\HelloWorldLambda\ package Amazon.Lambda.Tools
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Once this is installed we need to modify our &lt;code&gt;.csproj&lt;/code&gt; file.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-4-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;As you can see the toolkit installs as a Reference. If we try to run the toolkit command:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;dotnet lambda -help
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-4-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;We don't get any help :( what we need to do is modify our &lt;code&gt;.csproj&lt;/code&gt; file to change the &lt;code&gt;PackageReference&lt;/code&gt; to &lt;code&gt;DotNetCliToolReference&lt;/code&gt;. &lt;/p&gt;

&lt;p&gt;Now the &lt;code&gt;.csproj&lt;/code&gt; file should look like:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;Project Sdk="Microsoft.NET.Sdk"&amp;gt;
  &amp;lt;PropertyGroup&amp;gt;
    &amp;lt;TargetFramework&amp;gt;netcoreapp1.0&amp;lt;/TargetFramework&amp;gt;
  &amp;lt;/PropertyGroup&amp;gt;
  &amp;lt;ItemGroup&amp;gt;
    &amp;lt;PackageReference Include="Amazon.Lambda.Serialization.Json" Version="1.0.1" /&amp;gt;
    &amp;lt;DotNetCliToolReference Include="Amazon.Lambda.Tools" Version="1.4.0" /&amp;gt;
    &amp;lt;PackageReference Include="Marten" Version="1.4.1" /&amp;gt;
  &amp;lt;/ItemGroup&amp;gt;
&amp;lt;/Project&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now when we save and run the command again:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-4-03.png" alt="" /&gt;&lt;/p&gt;

&lt;h1&gt;Publishing&lt;/h1&gt;

&lt;p&gt;Publishing is very similar to pushing using dotnet clr, except we need to run the command from the project folder we need to publish so that we can have access to the tools. &lt;/p&gt;

&lt;p&gt;Note: this may be different using s solution, but I haven't tested it.&lt;/p&gt;

&lt;p&gt;Let's run:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;dotnet lambda package -c Release -o ../HelloWorldLambda.zip -f netcoreapp1.0
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This should create a zip file in the root directory where we published the first sample.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-4-04.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now we can go back into AWS and upload and configure the Lambda. In part 5 we will configure the VPC so we can connect to RDS and test our queries.&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2017/03/18/part3-saving-retrieving-data-using-marten/</guid><link>http://www.philliphaydon.com/2017/03/18/part3-saving-retrieving-data-using-marten/</link><a10:author><a10:name /></a10:author><category>aws</category><category>dotnetcore</category><category>lambda</category><category>marten</category><category>postgresql</category><title>Saving / Retrieving data using Marten</title><description>&lt;p&gt;So far we have created a basic lambda, deployed it into AWS, and executed it. However, it doesn't do anything useful. While there are many things you can do, one of the things you're most likely going to want to do is touch a database. But let's get a little bit fancy and do it with Marten.&lt;/p&gt;

&lt;p&gt;If you don't know Marten, you're missing out. It's a LOT of fun. There's two parts to Marten, an event store, and the super awesome fun bit that I love. The Document Database. &lt;/p&gt;

&lt;p&gt;Marten uses PostgreSQL's fancy JSONB data type to store documents in the database, with an API built on top to make working with your domain easier without having to worry about your schema. Check it out.&lt;/p&gt;

&lt;p&gt;&lt;a href="http://jasperfx.github.io/marten/"&gt;http://jasperfx.github.io/marten/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Let's get started. &lt;/p&gt;

</description><pubDate>Fri, 17 Mar 2017 16:00:00 Z</pubDate><a10:updated>2017-03-17T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;So far we have created a basic lambda, deployed it into AWS, and executed it. However, it doesn't do anything useful. While there are many things you can do, one of the things you're most likely going to want to do is touch a database. But let's get a little bit fancy and do it with Marten.&lt;/p&gt;

&lt;p&gt;If you don't know Marten, you're missing out. It's a LOT of fun. There's two parts to Marten, an event store, and the super awesome fun bit that I love. The Document Database. &lt;/p&gt;

&lt;p&gt;Marten uses PostgreSQL's fancy JSONB data type to store documents in the database, with an API built on top to make working with your domain easier without having to worry about your schema. Check it out.&lt;/p&gt;

&lt;p&gt;&lt;a href="http://jasperfx.github.io/marten/"&gt;http://jasperfx.github.io/marten/&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Let's get started. &lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;h1&gt;Installing / Configuring Marten&lt;/h1&gt;

&lt;p&gt;Marten only works with PostgreSQL, so I let's assume you've got that installed. Let's start by installing Marten into the hello world app. We run the dotnet command from the command line / terminal to install the package into the HelloWorldLambda project.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;gt; dotnet add .\HelloWorldLambda\ package Marten
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now we can add the Marten reference&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;using Marten;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Next let's create a constructor and create a document store inside the constructor.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;private readonly IDocumentStore store;
public Handler()
{
    store =DocumentStore.For(_ =&amp;gt;
    {
        _.Connection("Host=...;Username=...;Password=...;Database=...");
        _.AutoCreateSchemaObjects = AutoCreate.CreateOrUpdate;
    });
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Too easy! Now we are ready to use Marten. &lt;/p&gt;

&lt;h1&gt;Implementing Save&lt;/h1&gt;

&lt;p&gt;In our handler, let's add 2 new methods. &lt;code&gt;Get&lt;/code&gt; and &lt;code&gt;Save&lt;/code&gt;&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;[LambdaSerializer(typeof(JsonSerializer))]
public Product Save(Product product){
    return null;
}

[LambdaSerializer(typeof(JsonSerializer))]
public Product Get(GetProductRequest request) {
    return null;
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Create the two new classes. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }      
    public string Description { get; set; }
}

public class GetProductRequest
{
    public int Id { get; set; }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Let's start with the save. Using the Marten Store we created in the constructor, let's create a session. The session is effectively a Unit of Work. No you don't need to wrap this up in your own glorified unit of work like everyone attempts to do with Entity Framework or NHibernate. Just leave it alone.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;[LambdaSerializer(typeof(JsonSerializer))]
public Product Save(Product product) {
    using (var session = store.LightweightSession()) {
        session.Store(product);
        session.SaveChanges();

        return product;
    }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;So we open a session. I'm using a Lightweight session, this is a session that does not contain any change tracking at all. Refer to the documentation for more information. &lt;/p&gt;

&lt;p&gt;We add the product that is passed in to the session, call Save Changes, then return the product back out. This will populate the Id field and we will know the Id of the newly inserted product.&lt;/p&gt;

&lt;p&gt;Next let's work with the Get request.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;[LambdaSerializer(typeof(JsonSerializer))]
public Product Get(GetProductRequest request) {
    using (var session = store.QuerySession()) {
        return session.Load&amp;lt;Product&amp;gt;(request.Id);
    }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;In this scenario we open only a Query session, this is a session which only allows you to query and does not provide change tracking or any ability to store an object. &lt;/p&gt;

&lt;p&gt;We load based on the Id and return the product back out.&lt;/p&gt;

&lt;p&gt;BAM! And we are done! That's Marten configured, and saving/retrieving data. Too easy!&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;It's worth noting that this is purely an example and obviously you would want to do more validation around the requests. &lt;/p&gt;

&lt;p&gt;Next up is building / configuring the new Lambda handler implementations.&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2017/03/16/part2-building-and-configuring-the-lambda-in-aws/</guid><link>http://www.philliphaydon.com/2017/03/16/part2-building-and-configuring-the-lambda-in-aws/</link><a10:author><a10:name /></a10:author><category>aws</category><category>dotnetcore</category><category>lambda</category><title>Building / Configuring the Lambda into AWS</title><description>&lt;p&gt;Now after part 1, we should have a basic lambda ready to build and deploy. Because this is such a basic handler, this will be published using the dotnet sdk, but if you have dependencies it's best to use the AWS Tools which I will show in part 4. &lt;/p&gt;

&lt;h1&gt;Publishing the project&lt;/h1&gt;

&lt;p&gt;Publishing is easy! All we need to do is run &lt;code&gt;publish&lt;/code&gt;&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;phill@DESKTOP-599M841 D:\CSharpLambdaSample\HelloWorldLambda
$ dotnet publish
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorldLambda -&amp;gt; D:\CSharpLambdaSample\HelloWorldLambda\bin\Debug\netcoreapp1.0\HelloWorldLambda.dll
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;We can change the output directory by an &lt;code&gt;-o&lt;/code&gt; argument, and change from debug to release passing in a &lt;code&gt;-c&lt;/code&gt; argument.&lt;/p&gt;

</description><pubDate>Wed, 15 Mar 2017 16:00:00 Z</pubDate><a10:updated>2017-03-15T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Now after part 1, we should have a basic lambda ready to build and deploy. Because this is such a basic handler, this will be published using the dotnet sdk, but if you have dependencies it's best to use the AWS Tools which I will show in part 4. &lt;/p&gt;

&lt;h1&gt;Publishing the project&lt;/h1&gt;

&lt;p&gt;Publishing is easy! All we need to do is run &lt;code&gt;publish&lt;/code&gt;&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;phill@DESKTOP-599M841 D:\CSharpLambdaSample\HelloWorldLambda
$ dotnet publish
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorldLambda -&amp;gt; D:\CSharpLambdaSample\HelloWorldLambda\bin\Debug\netcoreapp1.0\HelloWorldLambda.dll
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;We can change the output directory by an &lt;code&gt;-o&lt;/code&gt; argument, and change from debug to release passing in a &lt;code&gt;-c&lt;/code&gt; argument.&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;pre&gt;&lt;code&gt;phill@DESKTOP-599M841 D:\CSharpLambdaSample\HelloWorldLambda
$ dotnet publish -o ./../published/ -c Release
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorldLambda -&amp;gt; D:\CSharpLambdaSample\HelloWorldLambda\bin\Release\netcoreapp1.0\HelloWorldLambda.dll
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Even tho the output is... in another folder, it does actually publish to the specified folder...&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now we can zip the contents of the directory. Don't zip the directory itself! You only want to zip the contents of the directory so that the .dll is in the root of the zip file. &lt;/p&gt;

&lt;p&gt;The name of the zip can be anything you want, you can version the zip during build process, or how ever you desire.&lt;/p&gt;

&lt;h1&gt;Configuring the Lambda&lt;/h1&gt;

&lt;p&gt;Next we can configure and upload the Lambda. You can do it via the AWS Tools, CLI, SDK's... However I'm only going to show how it's done using the AWS Console.&lt;/p&gt;

&lt;p&gt;From the Lambda Dashboard, click &lt;code&gt;Create a Lambda Function&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Next click on &lt;code&gt;Blank Function&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-03.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Click on &lt;code&gt;Next&lt;/code&gt; from the Configure Trigger. We don't need a trigger at this point, that's another topic. It allows you to create the Lambda as a cron job, or trigger from a notification, etc. We just want to test awesome C#iness.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-04.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now we can configure the Lambda itself.&lt;/p&gt;

&lt;p&gt;Give it a name, I called this &lt;code&gt;hello-world-csharp&lt;/code&gt;, set the runtime to &lt;code&gt;C#&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-05.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Next you need to upload the package, so select the zip file we made earlier.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-06.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Lastly we need to configure the Handler and Role.&lt;/p&gt;

&lt;p&gt;The handler is important, it has a defined structure to it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Handler:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Handler is split into 3 parts, separated by ::&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;ASSEMBLY :: NAMESPACE :: METHOD&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;So we have:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Assembly = &lt;code&gt;HelloWorldLambda&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Namespace = &lt;code&gt;HelloWorldLambda.Handler&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Method = &lt;code&gt;Handle&lt;/code&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;So our Handler should be configured like:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;HelloWorldLambda::HelloWorldLambda.Handler::Handle&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Role:&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;If you've never used an AWS Lambda's before, you need to give it permissions. So select &lt;code&gt;Create new role from templates(s)&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Add a new name, like &lt;code&gt;basic-csharp-lambda-role&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;And for Policy templates, just select &lt;code&gt;Simple Microservice permissions&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-07.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Finally, hit &lt;code&gt;Next&lt;/code&gt; and verify the new function, and press &lt;code&gt;Create function&lt;/code&gt;&lt;/p&gt;

&lt;h1&gt;Testing the Lambda&lt;/h1&gt;

&lt;p&gt;Now that we have created the lambda, it's ready to test!!!! The fun part, finally.&lt;/p&gt;

&lt;p&gt;Click on &lt;code&gt;Actions&lt;/code&gt; and select &lt;code&gt;Configure test event&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-08.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;We don't need to use a template, so let's just modify the Hello World example. Change the default:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-09.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;To have some test data that matches our &lt;code&gt;Request&lt;/code&gt; model, so just add a single property called &lt;code&gt;name&lt;/code&gt; with a value.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-10.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Now you can hit &lt;code&gt;Save and test&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;You should get a loading screen with... &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-2-11.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;BOOM! We have created an AWS C# Lambda in VSCode, published it and deployed it to AWS Lambda, configured it, and tested it.&lt;/p&gt;

&lt;p&gt;Next up we will do something a little more fancy and query the database!&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2017/03/15/part1-creating-a-good-old-hello-world-aws-csharp-lambda/</guid><link>http://www.philliphaydon.com/2017/03/15/part1-creating-a-good-old-hello-world-aws-csharp-lambda/</link><a10:author><a10:name /></a10:author><category>aws</category><category>dotnetcore</category><category>lambda</category><title>Creating a good old Hello World AWS C# Lambda</title><description>&lt;p&gt;So finally .NET Core Tools has reached 1.0 and after playing around... it's actually usable now. I can do all the things easily, that I once found difficult.&lt;/p&gt;

&lt;p&gt;This blog series is about creating a basic sample handler for an AWS C# Lambda, and extending it to configure a VPC, and query PostgreSQL using Marten, all using VSCode.&lt;/p&gt;

&lt;p&gt;To follow this series you need to install:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;.NET Core SDK from &lt;a href="https://www.microsoft.com/net/core#windowscmd"&gt;Microsoft&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/"&gt;VSCode&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://marketplace.visualstudio.com/items?itemName=ms-vscode.csharp"&gt;C#&lt;/a&gt; for VSCode&lt;/li&gt;
&lt;li&gt;&lt;a href="https://marketplace.visualstudio.com/items?itemName=jchannon.csharpextensions"&gt;C# Extensions&lt;/a&gt;... Extension for VSCode&lt;/li&gt;
&lt;/ul&gt;

&lt;h1&gt;Creating the class library&lt;/h1&gt;

&lt;p&gt;Let's start by creating a class library, create a folder some where, and navigate to it in terminal or command line. Then run the command: &lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;dotnet new classlib -n HelloWorldLambda -f netcoreapp1.0&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It's important you target netcoreapp1.0 as this is what's supported on AWS C# Lambda's right now.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;phill@DESKTOP-599M841 D:\CSharpLambdaSample
$ dotnet new classlib -n HelloWorldLambda -f netcoreapp1.0
Content generation time: 22.1887 ms
The template "Class library" created successfully.
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now you can open up the folder in VSCode.&lt;/p&gt;

</description><pubDate>Tue, 14 Mar 2017 16:00:00 Z</pubDate><a10:updated>2017-03-14T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;So finally .NET Core Tools has reached 1.0 and after playing around... it's actually usable now. I can do all the things easily, that I once found difficult.&lt;/p&gt;

&lt;p&gt;This blog series is about creating a basic sample handler for an AWS C# Lambda, and extending it to configure a VPC, and query PostgreSQL using Marten, all using VSCode.&lt;/p&gt;

&lt;p&gt;To follow this series you need to install:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;.NET Core SDK from &lt;a href="https://www.microsoft.com/net/core#windowscmd"&gt;Microsoft&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://code.visualstudio.com/"&gt;VSCode&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://marketplace.visualstudio.com/items?itemName=ms-vscode.csharp"&gt;C#&lt;/a&gt; for VSCode&lt;/li&gt;
&lt;li&gt;&lt;a href="https://marketplace.visualstudio.com/items?itemName=jchannon.csharpextensions"&gt;C# Extensions&lt;/a&gt;... Extension for VSCode&lt;/li&gt;
&lt;/ul&gt;

&lt;h1&gt;Creating the class library&lt;/h1&gt;

&lt;p&gt;Let's start by creating a class library, create a folder some where, and navigate to it in terminal or command line. Then run the command: &lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;dotnet new classlib -n HelloWorldLambda -f netcoreapp1.0&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;It's important you target netcoreapp1.0 as this is what's supported on AWS C# Lambda's right now.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;phill@DESKTOP-599M841 D:\CSharpLambdaSample
$ dotnet new classlib -n HelloWorldLambda -f netcoreapp1.0
Content generation time: 22.1887 ms
The template "Class library" created successfully.
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now you can open up the folder in VSCode.&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-1-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Let's rename the class to Handler, and add 2 new classes, Result and Request. You can use the C# Extensions to right click on the folder and add a new class.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/part-1-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;After adding the classes we can add a &lt;code&gt;Handle&lt;/code&gt; method to the Handler class that takes a Request and returns the Result.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;public class Handler
{
    public Result Handle(Request request) {
        return new Result {
            HelloWorld = request.Name
        };
    }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;We can now fill in the Request / Result classes with the 2 properties.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;public class Request
{
    public string Name { get; set; }
}

public class Result
{
    public string HelloWorld { get; set; }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now if we build this, it should, just build.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;PS D:\CSharpLambdaSample&amp;gt; cd .\HelloWorldLambda\
PS D:\CSharpLambdaSample\HelloWorldLambda&amp;gt; dotnet restore
  Restoring packages for D:\CSharpLambdaSample\HelloWorldLambda\HelloWorldLambda.cs proj...
  Generating MSBuild file D:\CSharpLambdaSample\HelloWorldLambda\obj\HelloWorldLambda.csproj.nuget.g.props.
  Writing lock file to disk. Path: D:\CSharpLambdaSample\HelloWorldLambda\obj\project.assets.json
  Restore completed in 340.03 ms for D:\CSharpLambdaSample\HelloWorldLambda\HelloWorldLambda.csproj.

  NuGet Config files used:
      C:\Users\phill\AppData\Roaming\NuGet\NuGet.Config
      C:\Program Files (x86)\NuGet\Config\Microsoft.VisualStudio.Offline.config

  Feeds used:
      https://www.nuget.org/api/v2/
      C:\Program Files (x86)\Microsoft SDKs\NuGetPackages\
PS D:\CSharpLambdaSample\HelloWorldLambda&amp;gt; dotnet build
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  HelloWorldLambda -&amp;gt; D:\CSharpLambdaSample\HelloWorldLambda\bin\Debug\netcoreapp1.0\HelloWorldLambda.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.86
PS D:\CSharpLambdaSample\HelloWorldLambda&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;The idea is that we will pass in a JSON Object to the AWS Lambda, it will take the &lt;code&gt;Name&lt;/code&gt; property and return a new object with a new property called &lt;code&gt;HelloWorld&lt;/code&gt; containing the value passed in.&lt;/p&gt;

&lt;p&gt;Super easy scenario.&lt;/p&gt;

&lt;h1&gt;Configuring the Lambda for serialization&lt;/h1&gt;

&lt;p&gt;So when the lambda is called we need it to deserialize the request to a &lt;code&gt;Request&lt;/code&gt; object and then serialize the &lt;code&gt;Result&lt;/code&gt; object when it returns.&lt;/p&gt;

&lt;p&gt;To do this we need to install the nuget package &lt;a href="https://www.nuget.org/packages/Amazon.Lambda.Serialization.Json/"&gt;Amazon.Lambda.Serialization.Json&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;From the terminal/command line, run:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;dotnet add .\HelloWorldLambda.csproj package Amazon.Lambda.Serialization.Json&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;I like to specify the project that it's to be added to, it's handy if you're in the parent directory and want to add to 1 of many projects.&lt;/p&gt;

&lt;p&gt;Running this should output something similar to:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;PS D:\CSharpLambdaSample\HelloWorldLambda&amp;gt; dotnet add .\HelloWorldLambda.csproj package Amazon.Lambda.Serialization.Json
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

Writing C:\Users\phill\AppData\Local\Temp\tmpFF0F.tmp
info : Adding PackageReference for package 'Amazon.Lambda.Serialization.Json' into project '.\HelloWorldLambda.csproj'.
log  : Restoring packages for D:\CSharpLambdaSample\HelloWorldLambda\HelloWorldLambda.csproj...
info :   GET https://www.nuget.org/api/v2/FindPackagesById()?id='Amazon.Lambda.Serialization.Json'
info :   OK https://www.nuget.org/api/v2/FindPackagesById()?id='Amazon.Lambda.Serialization.Json' 991ms
info :   GET https://www.nuget.org/api/v2/package/Amazon.Lambda.Serialization.Json/1.0.1
info :   OK https://www.nuget.org/api/v2/package/Amazon.Lambda.Serialization.Json/1.0.1 265ms
info : Package 'Amazon.Lambda.Serialization.Json' is compatible with all the specified frameworks in project '.\HelloWorldLambda.csproj'.
info : PackageReference for package 'Amazon.Lambda.Serialization.Json' version '1.0.1' added to file 'D:\CSharpLambdaSample\HelloWorldLambda\HelloWorldLambda.csproj'.PS D:\CSharpLambdaSample\HelloWorldLambda&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;You can run &lt;code&gt;dotnet restore&lt;/code&gt; after too.&lt;/p&gt;

&lt;p&gt;If you look at the csproj file you should now have a package reference.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;Project Sdk="Microsoft.NET.Sdk"&amp;gt;
  &amp;lt;PropertyGroup&amp;gt;
    &amp;lt;TargetFramework&amp;gt;netcoreapp1.0&amp;lt;/TargetFramework&amp;gt;
  &amp;lt;/PropertyGroup&amp;gt;
  &amp;lt;ItemGroup&amp;gt;
    &amp;lt;PackageReference Include="Amazon.Lambda.Serialization.Json" Version="1.0.1" /&amp;gt;
  &amp;lt;/ItemGroup&amp;gt;
&amp;lt;/Project&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Next you can reference the following into the handler class:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.Json;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now on the handler method we need to add the &lt;code&gt;LambdaSerializer&lt;/code&gt; attribute to the method:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;[LambdaSerializer(typeof(JsonSerializer))]
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Your final file should look like:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;using System;
using Amazon.Lambda.Core;
using Amazon.Lambda.Serialization.Json;

namespace HelloWorldLambda
{
    public class Handler
    {
        [LambdaSerializer(typeof(JsonSerializer))]
        public Result Handle(Request request) {
            return new Result {
                HelloWorld = request.Name
            };
        }
    }
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Running &lt;code&gt;dotnet build&lt;/code&gt; should be successful.&lt;/p&gt;

&lt;p&gt;In part 2 we will publish, deploy, and test the lambda.&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2016/05/07/postgresql-and-its-array-datatype/</guid><link>http://www.philliphaydon.com/2016/05/07/postgresql-and-its-array-datatype/</link><a10:author><a10:name /></a10:author><category>PostgreSQL</category><title>PostgreSQL and it's Array data type</title><description>&lt;p&gt;Switching between PostgreSQL and SQL Server, I've only ever really used the features of SQL Server in PostgreSQL. &lt;/p&gt;

&lt;p&gt;Lately after discovering Marten, I've been exploring features of PostgreSQL.&lt;/p&gt;

&lt;p&gt;Array data types, are the new awesome (today as of writing this, for me)&lt;/p&gt;

&lt;h2&gt;Defining the table&lt;/h2&gt;

&lt;p&gt;Using a really boring example, of a table, with an id, and an array of values.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;drop table if exists test_arrays;

create table test_arrays (
    id int,
    values int[]
);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Above you can see the values is declared as an array type by adding &lt;code&gt;[]&lt;/code&gt; to the end of &lt;code&gt;int&lt;/code&gt;, this declares it as an int array.&lt;/p&gt;

</description><pubDate>Fri, 06 May 2016 16:00:00 Z</pubDate><a10:updated>2016-05-06T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Switching between PostgreSQL and SQL Server, I've only ever really used the features of SQL Server in PostgreSQL. &lt;/p&gt;

&lt;p&gt;Lately after discovering Marten, I've been exploring features of PostgreSQL.&lt;/p&gt;

&lt;p&gt;Array data types, are the new awesome (today as of writing this, for me)&lt;/p&gt;

&lt;h2&gt;Defining the table&lt;/h2&gt;

&lt;p&gt;Using a really boring example, of a table, with an id, and an array of values.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;drop table if exists test_arrays;

create table test_arrays (
    id int,
    values int[]
);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Above you can see the values is declared as an array type by adding &lt;code&gt;[]&lt;/code&gt; to the end of &lt;code&gt;int&lt;/code&gt;, this declares it as an int array.&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;p&gt;PostgreSQL also supports multi-dimentional arrays by adding more square brackets to the end, for example we could define a multi dimentional array of &lt;code&gt;text&lt;/code&gt; values by declaring the column as &lt;code&gt;text[][]&lt;/code&gt; which would make it two dimentional.&lt;/p&gt;

&lt;p&gt;It also supports fixed size arrays, by passing an integer to the like &lt;code&gt;text[5]&lt;/code&gt;, this would make the array length of 5. &lt;/p&gt;

&lt;p&gt;I'm not going to discuss fixed length or multidimentional arrays in this post tho.&lt;/p&gt;

&lt;h2&gt;Inserting Data&lt;/h2&gt;

&lt;p&gt;Inserting data is a little strange...&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;insert into test_arrays (id, values) values (1, '{1, 5, 9}');
insert into test_arrays (id, values) values (2, '{2, 5, 20}');
insert into test_arrays (id, values) values (3, '{8, 6, 50}');
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Here I'm inserting 3 rows, as you can see the arrays are inserted by passing a string value that contains an array, wrapped in curly brackets. &lt;/p&gt;

&lt;p&gt;When inserting string values, they need to be wrapped in double quotes, for example:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;'{"Banana", "Apple", "Watermelon"}'
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This would be a value array of strings for use in &lt;code&gt;text[]&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;An alternative approach is to use the &lt;code&gt;array&lt;/code&gt; type passing in the values to the constructor like so:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;insert into test_arrays (id, values) values (4, array[12, 7, 3]);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;or&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;insert into test_arrays (id, values) values (4, array["Banana", "Apple", "Watermelon"]);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;I prefer the string approach.&lt;/p&gt;

&lt;h2&gt;Filtering data&lt;/h2&gt;

&lt;p&gt;So there's probably 3 main scenarios I can think of when filtering an array.&lt;/p&gt;

&lt;p&gt;Given the data we inserted above&lt;/p&gt;

&lt;h4&gt;Does the array contain the value 5?&lt;/h4&gt;

&lt;pre&gt;&lt;code&gt;-- does the column contain 5?
select * from test_arrays where 5 = any(values);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This says, does the value of 5 equal any of the values in the array. It's a little strange when you write it the first time but it does make sense.&lt;/p&gt;

&lt;p&gt;This would result in rows 1 and 2 being returned.&lt;/p&gt;

&lt;h4&gt;Does the array contain atleast 1 of the values from another array?&lt;/h4&gt;

&lt;pre&gt;&lt;code&gt;-- does the column contain at-least one of
select * from test_arrays where values &amp;amp;&amp;amp; array[6, 9];
select * from test_arrays where values &amp;amp;&amp;amp; '{6, 9}'::int[];
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This uses the &amp;amp;&amp;amp; operator which is 'overlap', in other words, does the (left) array contain any of the elements in the (right) array.&lt;/p&gt;

&lt;p&gt;This would result in rows 1 and 3 being returned.&lt;/p&gt;

&lt;p&gt;I wrote this using both ways of writing the array.&lt;/p&gt;

&lt;h4&gt;Does the array NOT contain any of the values from another array?&lt;/h4&gt;

&lt;p&gt;Very similar to the contains, only we specify it as &lt;code&gt;not&lt;/code&gt; before the condition.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;-- does the column NOT contain at-least one of
select * from test_arrays where not values &amp;amp;&amp;amp; array[6, 9];
select * from test_arrays where not values &amp;amp;&amp;amp; '{6, 9}'::int[];
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This would result in 1 row being returned.&lt;/p&gt;

&lt;h4&gt;Does the array contain all of the values from another array?&lt;/h4&gt;

&lt;pre&gt;&lt;code&gt;-- does the column contain all of
select * from test_arrays where values @&amp;gt; array[6, 9];
select * from test_arrays where values @&amp;gt; '{6, 9}'::int[];
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This uses the contains operator, so we're saying, given the (left) array of values, does it contain all of the values in the (right) array values.&lt;/p&gt;

&lt;p&gt;This would result in row 1 being returned.&lt;/p&gt;

&lt;h2&gt;Returning data&lt;/h2&gt;

&lt;p&gt;If you're selecting out your array, it will be selected as the format that we inserted it as.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/postgresql-arrays-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;This makes parsing it a little hard. In C# you would get a string back, and have to strip off the &lt;code&gt;{ }&lt;/code&gt; and then split on &lt;code&gt;,&lt;/code&gt;. The good news is there are options to help you return the data a little nicer.&lt;/p&gt;

&lt;h4&gt;array_to_string to remove curly brackets&lt;/h4&gt;

&lt;p&gt;Using &lt;code&gt;array_to_string(...)&lt;/code&gt; function, we can return the results as a string without the brackets.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;-- selecting the array as a string
select id, array_to_string(values, ',') from test_arrays;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This will return the same result as above, without the &lt;code&gt;{ }&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/postgresql-arrays-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;There's also the opposite, &lt;code&gt;string_to_array&lt;/code&gt;, which will allow you to pass a string in and get an array back.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;-- returning the array as a string
select string_to_array('9, 8, 7, 6', ',');
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This would give you:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/postgresql-arrays-03.png" alt="" /&gt;&lt;/p&gt;

&lt;h4&gt;unnest to return an array as rows&lt;/h4&gt;

&lt;p&gt;This is so awesome, trying to do this in SQL Server is SUCH a pain. PostgreSQL supports &lt;code&gt;unnest&lt;/code&gt; which will return each value of an array, as a row, it will also duplicate all data from other rows to each row returned for the array.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;-- return array column as rows
select id, unnest(values) from test_arrays
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This results in the following result:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/postgresql-arrays-04.png" alt="" /&gt;&lt;/p&gt;

&lt;h2&gt;Indexes &amp;amp; Doco&lt;/h2&gt;

&lt;p&gt;There's a lot of other stuff that I haven't included in this post. The type is Indexable, so you could apply a GIN index against the column for faster searching against &lt;code&gt;=&lt;/code&gt;, &lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt; and &lt;code&gt;&amp;lt;@&lt;/code&gt; / &lt;code&gt;&amp;gt;@&lt;/code&gt; operators.&lt;/p&gt;

&lt;p&gt;&lt;a href="http://www.postgresql.org/docs/9.5/static/gin-builtin-opclasses.html" title="http://www.postgresql.org/docs/9.5/static/gin-builtin-opclasses.html"&gt;http://www.postgresql.org/docs/9.5/static/gin-builtin-opclasses.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The PostgreSQL documentation is awesome. Check it out.  &lt;/p&gt;

&lt;p&gt;&lt;a href="http://www.postgresql.org/docs/9.5/static/functions-array.html" title="http://www.postgresql.org/docs/9.5/static/functions-array.html"&gt;http://www.postgresql.org/docs/9.5/static/functions-array.html&lt;/a&gt;&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2016/01/03/using-aws-api-gateway-and-sqs-to-recieve-sendgrid-event-hooks/</guid><link>http://www.philliphaydon.com/2016/01/03/using-aws-api-gateway-and-sqs-to-recieve-sendgrid-event-hooks/</link><a10:author><a10:name /></a10:author><category>API Gateway</category><category>AWS</category><category>Event Hooks</category><category>Sendgrid</category><category>SQS</category><title>Using AWS's API Gateway to recieve Sendgrid event hooks and shove them in SQS</title><description>&lt;p&gt;Sendgrid hooks allow you to recieve notifications of events that occur to the emails that you send, such as, was the email clicked, was it bounced, was it unsubscribed, etc.&lt;/p&gt;

&lt;p&gt;A list of events can be found &lt;a href="http://www.philliphaydon.com/0"&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;When sendgrid send the events, they send an array of events, so it may not be ideal to process those as they come in, and rather just accept them and process them when you can. To do that you can use AWS's API Gateway to accept the request and push the JSON body to SQS. &lt;/p&gt;

&lt;p&gt;Steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Setting up the Queue&lt;/li&gt;
&lt;li&gt;Setting up the Policy&lt;/li&gt;
&lt;li&gt;Setting up the Role&lt;/li&gt;
&lt;li&gt;Creating the API Gateway&lt;/li&gt;
&lt;li&gt;Test with Console&lt;/li&gt;
&lt;li&gt;Test with Postman&lt;/li&gt;
&lt;/ol&gt;

</description><pubDate>Sat, 02 Jan 2016 16:00:00 Z</pubDate><a10:updated>2016-01-02T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Sendgrid hooks allow you to recieve notifications of events that occur to the emails that you send, such as, was the email clicked, was it bounced, was it unsubscribed, etc.&lt;/p&gt;

&lt;p&gt;A list of events can be found &lt;a href="http://www.philliphaydon.com/0"&gt;here&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;When sendgrid send the events, they send an array of events, so it may not be ideal to process those as they come in, and rather just accept them and process them when you can. To do that you can use AWS's API Gateway to accept the request and push the JSON body to SQS. &lt;/p&gt;

&lt;p&gt;Steps:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Setting up the Queue&lt;/li&gt;
&lt;li&gt;Setting up the Policy&lt;/li&gt;
&lt;li&gt;Setting up the Role&lt;/li&gt;
&lt;li&gt;Creating the API Gateway&lt;/li&gt;
&lt;li&gt;Test with Console&lt;/li&gt;
&lt;li&gt;Test with Postman&lt;/li&gt;
&lt;/ol&gt;

&lt;!--excerpt--&gt;

&lt;h2&gt;Setting up the Queue&lt;/h2&gt;

&lt;p&gt;First up we need to create a new Queue, so in the AWS Console navigate to the SQS Service, and click &lt;code&gt;Create New Queue&lt;/code&gt;. I'm calling my queue &lt;code&gt;SendgridEvents&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Once created, click on the queue and look at the details.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Make note of the ARN value, we need that in the policy.&lt;/p&gt;

&lt;h2&gt;Setting up a new Policy&lt;/h2&gt;

&lt;p&gt;Next head on over to the IAM Service and navigate to Policies. We want to create a new policy, select &lt;code&gt;Create Your Own Policy&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Name the policy and add the following to the Policy Document:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Resource": [
                "arn:aws:sqs:us-west-2:1111222233334444:SendgridEvents"
            ],
            "Action": [
                "sqs:SendMessage"
            ]
        }
    ]
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This says we want to allow the &lt;code&gt;sqsSendMessage&lt;/code&gt; only for the &lt;code&gt;SendgridEvents&lt;/code&gt; queue we just created. This means that the Role we create will not allowed to be do anything else. &lt;/p&gt;

&lt;p&gt;Note: the &lt;code&gt;1111222233334444&lt;/code&gt; value should be your account id that you will see in your own ARN, you don't need to change this value.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-03.png" alt="" /&gt;&lt;/p&gt;

&lt;h2&gt;Setting up a new Role&lt;/h2&gt;

&lt;p&gt;Staying in IAM, navigate to Roles and select &lt;code&gt;Create New Role&lt;/code&gt;. &lt;/p&gt;

&lt;p&gt;Enter a Role Name&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-04.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Select from the &lt;code&gt;AWS Service Roles&lt;/code&gt; list, &lt;code&gt;Amazon EC2&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-05.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;This should skip over the &lt;code&gt;Establish Trust&lt;/code&gt;, we can't navigate back to it, but we will fix that shortly.&lt;/p&gt;

&lt;p&gt;Attach your policy that you just created.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-06.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Your policy review should look something like this.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-07.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Make note of the Role ARN, we need this for the API Gateway. &lt;/p&gt;

&lt;p&gt;Hit Create Role, and we're done... almost... &lt;/p&gt;

&lt;p&gt;Find your newly created role.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-08.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Click on it to select it, and then click on &lt;code&gt;Trust Relationships&lt;/code&gt;. &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-09.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Change the &lt;code&gt;Service&lt;/code&gt; to &lt;code&gt;apigateway.amazonaws.com&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-10.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;And then hit Update Trust Policy. &lt;/p&gt;

&lt;p&gt;Now we have setup a Queue, a SendMessage Policy, and a Role for use on the API Gateway service. &lt;/p&gt;

&lt;h2&gt;Creating the API Gateway&lt;/h2&gt;

&lt;p&gt;Navigate to the API Gateway Service and click &lt;code&gt;Create API&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Give it a name and description&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-11.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Because this API's sole purpose is to only accept requests from Sendgrid, I wont create any resources. Just a method.&lt;/p&gt;

&lt;p&gt;Click &lt;code&gt;Create Method&lt;/code&gt;, and select &lt;code&gt;POST&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-12.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;In the setup we want to click &lt;code&gt;Show advanced&lt;/code&gt; in the Integration type list. This allows us to select &lt;code&gt;AWS Service Proxy&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-14.png" alt="" /&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;AWS Region: The region must match the region the queue was created in&lt;/li&gt;
&lt;li&gt;AWS Service: SQS&lt;/li&gt;
&lt;li&gt;HTTP method: Since we are accepting JSON we need a POST for large requests&lt;/li&gt;
&lt;li&gt;Action Type: Use path override
&lt;ul&gt;
&lt;li&gt;Path override: This is the last portion of the ARN we copied when creating the Queue. It should be your ID + Queue Name. i.e &lt;code&gt;1111222233334444:SendgridEvents&lt;/code&gt;&lt;/li&gt;
&lt;li&gt;Execution role: This it the Role ARN we just created&lt;/li&gt;
&lt;/ul&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Click Save. Next we want to go to Integration Request.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-15.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Under HTTP Headers, add a new header for &lt;code&gt;Content-Type&lt;/code&gt; and set the value to &lt;code&gt;'application/x-www-form-urlencoded'&lt;/code&gt; (including the single quotes)&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-16.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;SQS only accepts two types of requests:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;GET: Where the values are passed via Query String&lt;/li&gt;
&lt;li&gt;POST: Where the values are passed as URL Encoded FORM Payload&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;But Sendgrid sends us JSON, so we need to parse the JSON into a URL Encoded FORM payload. &lt;/p&gt;

&lt;p&gt;If you don't change the Content-Type, then SQS will reject the request because the Content-Type is incorrect.&lt;/p&gt;

&lt;p&gt;Hit the little Circle Tick on the right hand side to save (bad UI)&lt;/p&gt;

&lt;p&gt;Next we want to add a Mapping Template.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-17.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Set the Content Type to &lt;code&gt;application/json&lt;/code&gt; this is what we will map from. Since Sendgrid will send us a JSON request, we need to map from that to the url encoded payload.&lt;/p&gt;

&lt;p&gt;After adding it, click it to change the mapping from &lt;code&gt;Input Passthrough&lt;/code&gt; to &lt;code&gt;Mapping template&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Set the template value to:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;Action=SendMessage&amp;amp;MessageBody=$util.urlEncode($util.escapeJavaScript($input.json('$')))
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-18.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Broken up this is 2 keys:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Action: SendMessage, this matches the Policy we created, it's the action we want to take on the Queue.&lt;/li&gt;
&lt;li&gt;MessageBody: This it the JSON payload from Sendgrid that we escape so we can send it to SQS.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Again we want to click that little Circle Tick icon to save.&lt;/p&gt;

&lt;h2&gt;Testing with the console&lt;/h2&gt;

&lt;p&gt;If you've followed along this far and understood everything, in theory it should all be configured correctly. So we will click &lt;code&gt;TEST&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-19.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;On this screen, click &lt;code&gt;Request Body&lt;/code&gt; and enter in some valid JSON.&lt;/p&gt;

&lt;p&gt;Sendgrid actually give you a sample of what they will send so we can use that for testing:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;[
  {
    "sg_message_id":"sendgrid_internal_message_id",
    "email": "john.doe@sendgrid.com",
    "timestamp": 1337966815,
    "event": "click",
    "url": "https://sendgrid.com",
    "userid": "1123",
    "template": "welcome"
  }
]
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-20.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Hit Test and it should send that JSON off to SQS and return a 200 status back:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-21.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;BAM it's working :)&lt;/p&gt;

&lt;h2&gt;Testing with Postman&lt;/h2&gt;

&lt;p&gt;From the same screen you can click Deploy API&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-22.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Select &lt;code&gt;New Stage&lt;/code&gt; and set a stage name. You can have many different stages for testing different things. We can create a test one for now and a production one later.&lt;/p&gt;

&lt;p&gt;Once it's deployed it will take you to a Stage Editor with an Invoke URL like so:&lt;/p&gt;

&lt;blockquote&gt;
  &lt;p&gt;Invoke URL: https://&lt;em&gt;snipped&lt;/em&gt;.execute-api.us-west-2.amazonaws.com/testing&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Taking the URL, we can fire up Postman, and enter in the URL that Amazon gave us:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-23.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Set the &lt;code&gt;Content-Type&lt;/code&gt; to &lt;code&gt;application/json&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-24.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Enter in the same JSON we entered for the Console test, and click &lt;code&gt;Send&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/sendgrid-api-25.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;And like the console, we get a result :)&lt;/p&gt;

&lt;p&gt;That's it!&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2015/10/21/gzip-with-script-cs-in-octopus-deploy/</guid><link>http://www.philliphaydon.com/2015/10/21/gzip-with-script-cs-in-octopus-deploy/</link><a10:author><a10:name /></a10:author><category>Octopus Deploy</category><category>scriptcs</category><title>GZip with scriptcs in Octopus Deploy</title><description>&lt;p&gt;I had this crazy problem tonight. I wanted to upload all my assets for my website into AWS S3, but I needed to gzip them first before sending them.&lt;/p&gt;

&lt;p&gt;Usually I do this in Team City with Grunt, but all my variable replacement is done in Octopus Deploy depending on the environment. &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Staging = S3 URLs&lt;/li&gt;
&lt;li&gt;Production = CloudFront URLs&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If the compression is done in Team City, variable replacement cannot happen on the GZipped file. &lt;/p&gt;

&lt;p&gt;I decided to move it to Octopus Deploy, my first thought was, could I run a node.js task in Octopus Deploy, the more I thought about it tho, I realised I don't want to install a dependency on node.js on the server.&lt;/p&gt;

&lt;p&gt;Second thought was to use Powershell... but I'm a real powershell no0b. &lt;/p&gt;

</description><pubDate>Tue, 20 Oct 2015 16:00:00 Z</pubDate><a10:updated>2015-10-20T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;I had this crazy problem tonight. I wanted to upload all my assets for my website into AWS S3, but I needed to gzip them first before sending them.&lt;/p&gt;

&lt;p&gt;Usually I do this in Team City with Grunt, but all my variable replacement is done in Octopus Deploy depending on the environment. &lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Staging = S3 URLs&lt;/li&gt;
&lt;li&gt;Production = CloudFront URLs&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If the compression is done in Team City, variable replacement cannot happen on the GZipped file. &lt;/p&gt;

&lt;p&gt;I decided to move it to Octopus Deploy, my first thought was, could I run a node.js task in Octopus Deploy, the more I thought about it tho, I realised I don't want to install a dependency on node.js on the server.&lt;/p&gt;

&lt;p&gt;Second thought was to use Powershell... but I'm a real powershell no0b. &lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;p&gt;However! Octopus Deploy has this little button!&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/octopus-deploy-script-cs-01.png" alt="New Step in Octopus Deploy showing 'C#' option" /&gt;&lt;/p&gt;

&lt;p&gt;OMG I can write... C# code... in my deployment process. &lt;/p&gt;

&lt;p&gt;SO I set out to read about &lt;code&gt;System.IO.Compression&lt;/code&gt; on &lt;a href="http://www.philliphaydon.com/0"&gt;MSDN&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;Working from this I came up with the following script:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;using System;
using System.IO;
using System.IO.Compression;

public static void Compress()
{
    var directory = Octopus.Parameters["Octopus.Action[Deploy Assets].Package.CustomInstallationDirectory"];

    Console.WriteLine("Trying to compress files in: " + directory);

    foreach (var fileToCompress in Directory.GetFiles(directory, "*.min.*", SearchOption.AllDirectories))
    {
        var fileInfo = new FileInfo(fileToCompress);

        if (!fileInfo.FullName.EndsWith(".min.js") &amp;amp;&amp;amp; !fileInfo.FullName.EndsWith(".min.css"))
        {
            continue;
        }

        var fileName = fileInfo.FullName.Replace(".min.js", "").Replace(".min.css", "") + ".gz" + fileInfo.Extension;

        using (FileStream originalFileStream = fileInfo.Open(FileMode.Open))
        using (FileStream compressedFileStream = File.Create(fileName))
        using (GZipStream compressionStream = new GZipStream(compressedFileStream, CompressionMode.Compress))
        {
            originalFileStream.CopyTo(compressionStream);

            Console.WriteLine("Compressed: " + fileInfo.FullName);
        }
    }
}

Compress();
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;I have a build step called &lt;strong&gt;Deploy Assets&lt;/strong&gt; which has a bunch of sub steps. This has a package which stores all the CSS/Images/JavaScript files that have been generated in Team City.&lt;/p&gt;

&lt;p&gt;In here I update the URLs in all the files, and then run a Compress JS, and finally upload them all to S3. &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/octopus-deploy-script-cs-02.png" alt="Octopus Deploy build steps" /&gt;&lt;/p&gt;

&lt;p&gt;Basically what this script does is iterates over all files in the deploy folder looking for anything that was already minified. &lt;/p&gt;

&lt;pre&gt;&lt;code&gt;if (!fileInfo.FullName.EndsWith(".min.js") &amp;amp;&amp;amp; !fileInfo.FullName.EndsWith(".min.css"))
{
    continue;
}
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;If it's not a &lt;code&gt;.min.js&lt;/code&gt; or &lt;code&gt;.min.css&lt;/code&gt; file, then it's ignored.&lt;/p&gt;

&lt;p&gt;It then creates a new &lt;code&gt;fileName&lt;/code&gt; stripping off the existing extension and appending &lt;code&gt;.gz&lt;/code&gt; followed by the last extension of the file. &lt;/p&gt;

&lt;p&gt;e.g &lt;code&gt;main.min.css&lt;/code&gt; becomes &lt;code&gt;main.gz.css&lt;/code&gt; &lt;/p&gt;

&lt;p&gt;Then GZips the file and saves it.&lt;/p&gt;

&lt;p&gt;Here's an example of the &lt;code&gt;main.js&lt;/code&gt; file minified and compressed:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/octopus-deploy-script-cs-03.png" alt="Example compressed file" /&gt;&lt;/p&gt;

&lt;p&gt;And that's all there is to it. You can basically write any C# code you want. &lt;/p&gt;

&lt;p&gt;Please refer to &lt;a href="http://scriptcs.net/"&gt;scriptcs website&lt;/a&gt; for more information on how to load dependencies and such.&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2014/08/16/permament-redirect-to-https-with-iis/</guid><link>http://www.philliphaydon.com/2014/08/16/permament-redirect-to-https-with-iis/</link><a10:author><a10:name /></a10:author><category>HTTPS</category><category>IIS</category><category>Nancy</category><title>Permament redirect to HTTPS with IIS</title><description>&lt;p&gt;Google has just recently updated their search results to give higher ranking to sites with an SSL Certificate, than to sites without, which is one of the best changes Google has made in recent years. There really is no excuse for not having a cert now. &lt;em&gt;(note, this is limited to small portion of sites but lets assume that this will be rolled out if Google proves it to be worth while)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="http://googleonlinesecurity.blogspot.co.uk/2014/08/https-as-ranking-signal_6.html"&gt;googleonlinesecurity - https-as-ranking-signal_6.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Unfortuntely for me it seems Github Pages does not support Certificates on custom domain names, yet... :( hopefully they will support this eventually so that I can avoid moving my blog.&lt;/p&gt;

&lt;p&gt;So one thing that pops up in the Nancy channel on JabbR every-now-n-then, is how to make all modules require SSL. &lt;/p&gt;

&lt;p&gt;From within a module you can call:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;this.RequiresHttps()
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;And this will push the site into &lt;code&gt;https&lt;/code&gt;, but having to write this everywhere is rather annoying, and during development on your local machine this is a bit of a hassle. Despite Visual Studio supporting an HTTPS url...&lt;/p&gt;

</description><pubDate>Fri, 15 Aug 2014 16:00:00 Z</pubDate><a10:updated>2014-08-15T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Google has just recently updated their search results to give higher ranking to sites with an SSL Certificate, than to sites without, which is one of the best changes Google has made in recent years. There really is no excuse for not having a cert now. &lt;em&gt;(note, this is limited to small portion of sites but lets assume that this will be rolled out if Google proves it to be worth while)&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;&lt;a href="http://googleonlinesecurity.blogspot.co.uk/2014/08/https-as-ranking-signal_6.html"&gt;googleonlinesecurity - https-as-ranking-signal_6.html&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Unfortuntely for me it seems Github Pages does not support Certificates on custom domain names, yet... :( hopefully they will support this eventually so that I can avoid moving my blog.&lt;/p&gt;

&lt;p&gt;So one thing that pops up in the Nancy channel on JabbR every-now-n-then, is how to make all modules require SSL. &lt;/p&gt;

&lt;p&gt;From within a module you can call:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;this.RequiresHttps()
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;And this will push the site into &lt;code&gt;https&lt;/code&gt;, but having to write this everywhere is rather annoying, and during development on your local machine this is a bit of a hassle. Despite Visual Studio supporting an HTTPS url...&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;If you didn't know this existed, you should. Basically in Visual Studio, select your Web Application project, then open up the properties window (don't right click and select properties, you need to open the window manually) in the window, set &lt;code&gt;SSL Enabled&lt;/code&gt; to true, this will add a SSL URL, then you can run the site and use that url instead of the non-https url.&lt;/p&gt;

&lt;hr /&gt;

&lt;p&gt;So during development you may want to develop without a certificate, but want your production site to be always using a certificate, this makes the &lt;code&gt;RequiresHttps()&lt;/code&gt; call a little annoying, we end up with conditional logic or something similar to figure out are we in development, or are we in production.&lt;/p&gt;

&lt;h2&gt;URL Rewite Extension for IIS&lt;/h2&gt;

&lt;p&gt;Download and install the &lt;a href="http://www.iis.net/downloads/microsoft/url-rewrite"&gt;URL Rewrite extension&lt;/a&gt; for IIS. Once installed we can do lots of fancy stuff for URL Rewriting, but we only want to handle the HTTPS redirect for this post.&lt;/p&gt;

&lt;p&gt;Now that it's installed, we can configure it one of two ways. If we use the GUI then we still need to copy the contents generated in the Web.config file and put that into a build script.&lt;/p&gt;

&lt;p&gt;Using the GUI, we can open up the URL Rewrite extension:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-02.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Using the GUI we can setup a new &lt;code&gt;Blank rule&lt;/code&gt; in the &lt;code&gt;Inbound rules&lt;/code&gt; section:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-03.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;Next we want to fill in the rule. &lt;/p&gt;

&lt;p&gt;For the Match URL, we want to use a super duper simple regular expression.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;.*&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;This basically says, match any character, as many times as you want.&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-04.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;For the Conditions, we want to check the Input &lt;code&gt;{HTTPS}&lt;/code&gt; match is off, if we have it on then we will end up in an infinate loop. eek&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-05.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;We don't care about Server Variables so skip that.&lt;/p&gt;

&lt;p&gt;And lastly is the action we want to take when the rule matches (that it's not an HTTPS request)&lt;/p&gt;

&lt;p&gt;We want to Redirect to:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;https://{HTTP_HOST}/{R:0}&lt;/code&gt;&lt;/p&gt;

&lt;p&gt;Tick append query string, and for the redirect type, select Permanment. &lt;/p&gt;

&lt;p&gt;&lt;code&gt;{HTTP_HOST}&lt;/code&gt; is the host name, so if your domain is &lt;code&gt;bananasareyellow.com&lt;/code&gt; then that's the value that will be put in the redirect url. You can hard code this value if you want, but if you want to support multiple host names then you will want to use the variable. &lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/https-iis-vs-06.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;After applying this new rule, when you visit your website with an HTTP url, it will be pushed into HTTPS. &lt;/p&gt;

&lt;h2&gt;Ahhh I've lost it!&lt;/h2&gt;

&lt;p&gt;All this information gets stored in your web.config file, and it looks like this:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;system.webServer&amp;gt;
    &amp;lt;rewrite&amp;gt;
        &amp;lt;rules&amp;gt;
            &amp;lt;rule name="HTTP to HTTPS redirect" stopProcessing="true"&amp;gt;
                &amp;lt;match url=".*" /&amp;gt;
                &amp;lt;conditions&amp;gt;
                    &amp;lt;add input="{HTTPS}" pattern="OFF" /&amp;gt;
                &amp;lt;/conditions&amp;gt;
                &amp;lt;action type="Redirect" url="https://{HTTP_HOST}/{R:0}" redirectType="Permanent" /&amp;gt;
            &amp;lt;/rule&amp;gt;
        &amp;lt;/rules&amp;gt;
    &amp;lt;/rewrite&amp;gt;
&amp;lt;/system.webServer&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;SO! &lt;/p&gt;

&lt;p&gt;Once you've written up your condition, you may want to roll this into your build script so that during development, that way when it's deployed it will always contain the rule and you wont lose it. &lt;/p&gt;

&lt;p&gt;You can also add a condition in here to say the rule does not apply when the site is a localhost or test domain, but in my opinion you shouldn't add test-checks in production code.&lt;/p&gt;
</a10:content></item><item><guid isPermaLink="true">http://www.philliphaydon.com/2014/06/09/taking-snapshots-of-a-video-and-displaying-it/</guid><link>http://www.philliphaydon.com/2014/06/09/taking-snapshots-of-a-video-and-displaying-it/</link><a10:author><a10:name /></a10:author><category>HTML5</category><category>JavaScript</category><title>Taking snap shots of a Video &amp; displaying it</title><description>&lt;p&gt;Now we've loaded an image/video, we've captured the drop event, we've displayed it...&lt;/p&gt;

&lt;p&gt;But what about taking screen grabs of a video?&lt;/p&gt;

&lt;p&gt;To start with I'll go back to the code we used in Part 1, so we have an input, a load button, and a video control:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;!DOCTYPE html&amp;gt;
&amp;lt;html&amp;gt;
&amp;lt;head&amp;gt;
    &amp;lt;meta charset="utf-8" /&amp;gt;
    &amp;lt;title&amp;gt;&amp;lt;/title&amp;gt;
&amp;lt;/head&amp;gt;
&amp;lt;body&amp;gt;      
    &amp;lt;div&amp;gt;
        &amp;lt;input type="file" id="video-input"&amp;gt;
        &amp;lt;input type="button" value="Load Selected Video" id="load-video" /&amp;gt;
    &amp;lt;/div&amp;gt;

    &amp;lt;div&amp;gt;
        &amp;lt;video id="video-container" controls&amp;gt;&amp;lt;/video&amp;gt;
    &amp;lt;/div&amp;gt;
&amp;lt;/body&amp;gt;
&amp;lt;/html&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Racing ahead we load that up:&lt;/p&gt;

</description><pubDate>Sun, 08 Jun 2014 16:00:00 Z</pubDate><a10:updated>2014-06-08T16:00:00Z</a10:updated><a10:content type="html">&lt;p&gt;Now we've loaded an image/video, we've captured the drop event, we've displayed it...&lt;/p&gt;

&lt;p&gt;But what about taking screen grabs of a video?&lt;/p&gt;

&lt;p&gt;To start with I'll go back to the code we used in Part 1, so we have an input, a load button, and a video control:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;!DOCTYPE html&amp;gt;
&amp;lt;html&amp;gt;
&amp;lt;head&amp;gt;
    &amp;lt;meta charset="utf-8" /&amp;gt;
    &amp;lt;title&amp;gt;&amp;lt;/title&amp;gt;
&amp;lt;/head&amp;gt;
&amp;lt;body&amp;gt;      
    &amp;lt;div&amp;gt;
        &amp;lt;input type="file" id="video-input"&amp;gt;
        &amp;lt;input type="button" value="Load Selected Video" id="load-video" /&amp;gt;
    &amp;lt;/div&amp;gt;

    &amp;lt;div&amp;gt;
        &amp;lt;video id="video-container" controls&amp;gt;&amp;lt;/video&amp;gt;
    &amp;lt;/div&amp;gt;
&amp;lt;/body&amp;gt;
&amp;lt;/html&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Racing ahead we load that up:&lt;/p&gt;

&lt;!--excerpt--&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;script&amp;gt;    
    (function(){

        var fileInput = document.getElementById('video-input');
        var video = document.getElementById('video-container');

        document.getElementById('load-video').addEventListener('click', function(){
            var fileUrl = window.URL.createObjectURL(fileInput.files[0]);

            video.src = fileUrl;            
        });

    })();   
&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Now that we have loaded our video, how do we capture screen grabs?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;&lt;em&gt;Note:&lt;/strong&gt; All sizes are halved becaused the test video is too large :)&lt;/em&gt;&lt;/p&gt;

&lt;h2&gt;The hidden canvas&lt;/h2&gt;

&lt;p&gt;The first thing we have to do is get the current data of the current frame being displayed. To do that we need to create a hidden canvas, and then draw onto it. &lt;/p&gt;

&lt;p&gt;So we're gonna add a canvas element to the UI:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;canvas id="temp-canvas" style="display:none"&amp;gt;&amp;lt;/canvas&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;The canvas doesn't need to display, it's simply there to capture data. Now we need to put the video into it, so lets add a button, lets also add somewhere to display the image once we have it. :&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;input type="button" value="Copy current frame to image below" id="load-image" /&amp;gt;

&amp;lt;br /&amp;gt;Image......&amp;lt;br /&amp;gt;
&amp;lt;img id="image-container" /&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;h2&gt;The fluff in between...&lt;/h2&gt;

&lt;p&gt;Now we need to add some fluff when the button to copy the current frame is clicked.&lt;/p&gt;

&lt;p&gt;Let's find the controls:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;var image = document.getElementById('image-container');
var canvas = document.getElementById('temp-canvas');
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;And add a click event to the new button:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;document.getElementById('load-image').addEventListener('click', function(){

});
&lt;/code&gt;&lt;/pre&gt;

&lt;h2&gt;Load that image!&lt;/h2&gt;

&lt;p&gt;What we need to do now is take the dom video control, as is. &lt;em&gt;(this means if you're using jQuery to find the element, you need to get the real dom element, not the jQuery wrapped element)&lt;/em&gt; &lt;/p&gt;

&lt;p&gt;This is currently our &lt;code&gt;video&lt;/code&gt; variable. &lt;/p&gt;

&lt;p&gt;First up we will size our canvas:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;canvas.height = video.videoHeight / 2;
canvas.width = video.videoWidth / 2;
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Then we need to create a context on our canvas. We will create a 2d context like do:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;var context = canvas.getContext("2d");
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This gives us a 2d rendering context whih we can use to draw on. We will draw the current video frame. Assuming we have pause the video in the middle somewhere, we can hit the button to capture the current frame by padding the video in like so:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight, 0, 0, canvas.width, canvas.height);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;&lt;strong&gt;// NOTE&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;For this I used the 9 property &lt;code&gt;drawImage&lt;/code&gt; function, this is:&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;drawImage(source, sourceStartX, sourceStartY, width, height, destinationStartX, destinationStartY, width, height) 
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;This means, the sourceStart is where you want the first pixel to be captured from, the width/height of the source from the start pixels. &lt;/p&gt;

&lt;p&gt;The destinationStart is where you want to place the image on the canvas and how you want it to stretch.&lt;/p&gt;

&lt;p&gt;My sample takes the actual height/width of the source and places it in the smaller canvas.&lt;/p&gt;

&lt;p&gt;You can also use&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;context.drawImage(video, 0, 0, video.videoWidth, video.videoHeight);
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Which will assume the same height/width for source/canvas.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;// END NOTE&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Now all that's left is to display it, to do this we call the &lt;code&gt;toDataURL()&lt;/code&gt; function defined on the canvas. This takes what's currently drawn and shows it in a image element.&lt;/p&gt;

&lt;pre&gt;&lt;code&gt;image.src = canvas.toDataURL();     
&lt;/code&gt;&lt;/pre&gt;

&lt;p&gt;Why do we do this rather than display the canvas itself? Well this allows you to right click the image and download it. :)&lt;/p&gt;

&lt;p&gt;Now we can load a video:&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/screen-capture-video-html5-01.png" alt="" /&gt;&lt;/p&gt;

&lt;p&gt;And then press the button to capture it!!!!&lt;/p&gt;

&lt;p&gt;&lt;img src="http://www.philliphaydon.com/images/screen-capture-video-html5-02.png" alt="" /&gt;&lt;/p&gt;

&lt;h2&gt;The final codez&lt;/h2&gt;

&lt;pre&gt;&lt;code&gt;&amp;lt;div&amp;gt;
    &amp;lt;input type="file" id="video-input"&amp;gt;
    &amp;lt;input type="button" value="Load Selected Video" id="load-video" /&amp;gt;
&amp;lt;/div&amp;gt;

&amp;lt;div&amp;gt;
    &amp;lt;br /&amp;gt;Video...&amp;lt;br /&amp;gt;
    &amp;lt;video id="video-container" controls&amp;gt;&amp;lt;/video&amp;gt;

    &amp;lt;br /&amp;gt;
    &amp;lt;input type="button" value="Copy current frame to image below" id="load-image" /&amp;gt;

    &amp;lt;br /&amp;gt;Image......&amp;lt;br /&amp;gt;
    &amp;lt;img id="image-container" /&amp;gt;
&amp;lt;/div&amp;gt;

&amp;lt;canvas id="temp-canvas" style="display:none"&amp;gt;&amp;lt;/canvas&amp;gt;

&amp;lt;script&amp;gt;
    (function(){

        var fileInput = document.getElementById('video-input');
        var video = document.getElementById('video-container');
        var image = document.getElementById('image-container');
        var canvas = document.getElementById('temp-canvas');

        document.getElementById('load-video').addEventListener('click', function(){
            var fileUrl = window.URL.createObjectURL(fileInput.files[0]);

            video.src = fileUrl;            
        });
        document.getElementById('load-image').addEventListener('click', function(){

            canvas.height = video.videoHeight;
            canvas.width = video.videoWidth;

            var context = canvas.getContext("2d");

            drawImage(source, sourceStartX, sourceStartY, width, height, destinationStartX, destinationStartY, width, height) 

            image.src = canvas.toDataURL();         
        });

    })();

&amp;lt;/script&amp;gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;h2&gt;What do we have...&lt;/h2&gt;

&lt;p&gt;So we've got an file &lt;code&gt;input&lt;/code&gt; control, it loads a video file into a &lt;code&gt;video&lt;/code&gt; element, then we scrub the video &amp;amp; hit that button, and bam, we capture what ever the current frame is.&lt;/p&gt;
</a10:content></item></channel></rss>