<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Danny Lane]]></title><description><![CDATA[Notes To _Self]]></description><link>https://dannylane.github.io</link><image><url>https://github.com/dannylane/dannylane.github.io/raw/master/images/zoom.png</url><title>Danny Lane</title><link>https://dannylane.github.io</link></image><generator>RSS for Node</generator><lastBuildDate>Mon, 09 Jul 2018 21:24:22 GMT</lastBuildDate><atom:link href="https://dannylane.github.io/rss/" rel="self" type="application/rss+xml"/><ttl>60</ttl><item><title><![CDATA[Azure Functions with ServiceBus and Blob Storage]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Serverless technologies are an area of cloud computing that I find very interesting, particularly the Function As A Service paradigm.</p>
</div>
<div class="paragraph">
<p>Products like <a href="https://aws.amazon.com/lambda/">AWS Lambda</a> and <a href="https://azure.microsoft.com/en-us/services/functions/">Azure Functions</a> have been around for a number of years now, and have had some pretty compelling use cases from the outset.</p>
</div>
<div class="paragraph">
<p>Everything from on prem cron jobs that need servers to live on, up to entire restful APIs are commonly finding new homes on these FAAS platforms.</p>
</div>
<div class="paragraph">
<p>Seeing features like <a href="https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/">Jobs</a> and <a href="https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/">CronJobs</a> in Kubernetes further backs up the feeling that that these 'write once, run occasionally' frameworks are have touched upon a genuine need in the industry and are not just an interesting technical solution looking for a problem.</p>
</div>
<div class="paragraph">
<p>I set up a pretty trivial data ingest pipeline to try out some of the different ways that Azure Functions interacts with some of the other cloud primitives offered by Microsoft.</p>
</div>
<div class="paragraph">
<p>The pipeline is made up of a number of azure functions, and some of these functions interact with <a href="https://azure.microsoft.com/en-us/services/service-bus/">Azure Service Bus</a>, <a href="https://azure.microsoft.com/en-us/services/storage/blobs/">Azure Blob Storage</a>, <a href="https://azure.microsoft.com/en-us/services/cosmos-db/">Azure CosmosDB</a> and plain old <a href="https://httpstatuses.com/418">HTTP</a>.</p>
</div>
<div class="paragraph">
<p>At a high level the pipeline has a trigger that initiates the flow with a Service Bus event. The second step downloads a csv with data on 5,000 movies and puts it to blob storage. In step 3 that csv is parsed and the data fired for as individual messages into another Service Bus queue. The last step in the pipeline ingests those messages with movie data and inserts them into a CosmosDB instance.</p>
</div>
<div class="paragraph">
<p>There is a 5th function that is exposed as a simple search API and is integrated with a static site served by Azure CDN.</p>
</div>
<div class="paragraph">
<p>In this first article I&#8217;m going to go through the first 2 steps, triggering the data flow and getting the CSV into Blob Storage.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_bindings">Bindings</h4>
<div class="paragraph">
<p>Defore I get into the code I should call out the <a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings">bindings and triggers</a> that Azure provides for interacting with Functions. There is a <a href="https://github.com/Azure/azure-functions-host/wiki/function.json">function.json</a> file that defines the configuration for a Function. The bindings sesiont of this json file tells Azure how the Function should be triggered and how it&#8217;s output should be treated. Theres a simple example here:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
   "bindings":[
      {
         "queueName":"testqueue",
         "connection":"MyServiceBusConnection",
         "name":"myQueueItem",
         "type":"serviceBusTrigger",
         "direction":"in"
      }
   ],
   "disabled":false
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Depending on SDK you are using (C# here but the Java SDK is very similar) the bindings into a Function can be autogenerated from attributes on the parameters. A future version of the runtime will remove the need for having the bindings section and the just the attributes will be used.</p>
</div>
<div class="paragraph">
<p>The bindings list <a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-triggers-bindings#supported-bindings">here</a> gives a good view as to what is supported for interacting with other Azure products. The HTTP and Timer are probably the two simplest bindings.</p>
</div>
<div class="paragraph">
<p>In my examples below I&#8217;m mostly working with HTTP, Service Bus, CosmosDB and Blob inputs/outputs.</p>
</div>
<div class="paragraph">
<p>Disclaimer, all these examples are trivial and contrived, none even resemble a prototype.</p>
</div>
</div>
<div class="sect3">
<h4 id="_trigger_movie_process">Trigger Movie Process</h4>
<div class="paragraph">
<p>.
<span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/seq1.png" alt="seq1.png"></span></p>
</div>
<div class="paragraph">
<p>The first Azure Function is triggered by a HTTP call and sends an event to Service Bus when the function is triggered.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-csharp" data-lang="csharp">[FunctionName("TriggerMovieProcess")]
public static IActionResult Run(
	[HttpTrigger(AuthorizationLevel.Anonymous, "get", Route = null)]HttpRequest req,
	[ServiceBus("TriggerMovieProcessQueue", Connection = "QueueConnectionString")] ICollector&lt;string&gt; outputServiceBus,
	TraceWriter log
)</code></pre>
</div>
</div>
<div class="paragraph">
<p><strong>Input</strong></p>
</div>
<div class="paragraph">
<p>The HTTPTrigger attribute on the req param tells Azure that this function should be exposed as a HTTP endpoint, the HttpRequest is available in the req object, allowing you full access to headers, form data, etc.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/invoke_http_function.png" alt="invoke http function.png"></span></p>
</div>
<div class="paragraph">
<p><strong>Output</strong></p>
</div>
<div class="paragraph">
<p>The ServiceBus attribute allows you to add messages to the Service Bus queue defined by the name and connection string in the attribute properties.</p>
</div>
<div class="paragraph">
<p>To get the service bus binding you need the following <a href="https://www.nuget.org/packages/Microsoft.Azure.WebJobs.ServiceBus/3.0.0-beta5">nuget</a> package</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-csharp" data-lang="csharp">string name = req.Query["name"];
outputServiceBus.Add(JsonConvert.SerializeObject(
	new StartMovieProcess
	{
		TriggeredBy = name,
		TriggeredOn = DateTime.UtcNow.ToString()
	}));

return name != null
? (ActionResult)new OkObjectResult($"Triggered by {name}")
: new BadRequestObjectResult("Send a name to trigger");</code></pre>
</div>
</div>
<div class="paragraph">
<p>I take a name from the query string and populate a StartMovieProcess object that I add to the queue.</p>
</div>
<div class="paragraph">
<p>The method returns a Http response to sattisfy the http request.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/trigger_movie_process_log.png" alt="trigger movie process log.png"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="_download_movie_data">Download movie data</h4>
<div class="paragraph">
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/seq2.png" alt="seq2.png"></span></p>
</div>
<div class="paragraph">
<p>The second Function in the pipeline is triggered by any message in the <em>TriggerMovieProcessQueue</em>.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/download_movie_data_log.png" alt="download movie data log.png"></span></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-csharp" data-lang="csharp">[FunctionName("DownloadMovieData")]
public static async Task Run(
	[ServiceBusTrigger("TriggerMovieProcessQueue", Connection = "QueueConnectionString")]string triggerEventJson,
	[BlobAttribute("moviedemo/%moviedata_file_name%", FileAccess.Write)] ICloudBlob blob,
	TraceWriter log)</code></pre>
</div>
</div>
<div class="paragraph">
<p>The ServiceBusTrigger Attribute tells Azure to invoke this function whenever messages are sent to the queue configured by the properties. The triggerEventJson will contain the data from the message. I could probably have used a strongly typed domain object here instead of flattened json but I haven&#8217;t tried it.</p>
</div>
<div class="paragraph">
<p>The output here is going to be a blob that I upload a file to.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-csharp" data-lang="csharp">StartMovieProcess triggerEvent = JsonConvert.DeserializeObject&lt;StartMovieProcess&gt;(triggerEventJson);

blob.Metadata.Add("TriggeredBy",triggerEvent.TriggeredBy);
blob.Metadata.Add("TriggeredOn",triggerEvent.TriggeredOn);

var uri = "https://github.com/yash91sharma/IMDB-Movie-Dataset-Analysis/blob/master/movie_metadata.csv?raw=true";

using (var httpClient = new HttpClient())
using (var responseStream = httpClient.GetStreamAsync(new Uri(uri)))
{
	await blob.UploadFromStreamAsync(responseStream.Result);
}

return;</code></pre>
</div>
</div>
<div class="paragraph">
<p>In the snippet above I grab the data from a CSV hosted on github and upload it to the blob on Azure Storage.</p>
</div>
<div class="paragraph">
<p>I added a couple of Metadata values to the blob form the message just to show how easy it is to push these attributes with the blob.</p>
</div>
<div class="paragraph">
<p>Once the file is uploaded this second function ends.</p>
</div>
<div class="paragraph">
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/blob_metadata_portal.png" alt="blob metadata portal.png"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="_setup_for_local_development">Setup for local development</h4>
<div class="paragraph">
<p>If you are using <a href="https://docs.microsoft.com/en-us/azure/azure-functions/functions-develop-vs">Visual Studio</a> you benefit from the tight developer experience that MS offer, in this article I&#8217;ll be using VS Code on Ubuntu.</p>
</div>
<div class="paragraph">
<p>The <a href="https://github.com/Microsoft/vscode-azurefunctions">Azure Function extension</a> for VSCode is definitely worth looking into.
<span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/functions_extension_vscode.png" alt="functions extension vscode.png"></span></p>
</div>
<div class="paragraph">
<p>To run the functions locally you will need the sdk <a href="https://www.nuget.org/packages/Microsoft.NET.Sdk.Functions">nuget</a> package and the Azure Functions <a href="https://github.com/Azure/azure-functions-core-tools">Core Tools</a> which offers a local runtime for Functions.</p>
</div>
</div>
<div class="sect3">
<h4 id="_running_the_functions">Running the Functions</h4>
<div class="paragraph">
<p>Running the functions on the local Function host is the same as running any other application once you have the prerequisites above setup.</p>
</div>
<div class="paragraph">
<div class="title">Who doesn&#8217;t love ascii art?</div>
<p><span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/host.png" alt="host.png"></span></p>
</div>
<div class="paragraph">
<p>The runtime gives some good logs around which Functions have been detected:
<span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/detecting_functions.png" alt="detecting functions.png"></span></p>
</div>
<div class="paragraph">
<p>URLs HTTP triggered Functions can be invoked from:
<span class="image"><img src="https://dannylane.github.io/images/azure_functions/part1/http_trigger_functions.png" alt="http trigger functions.png"></span></p>
</div>
<div class="paragraph">
<p>That’s all I&#8217;m going to cover in this part, in the next article I&#8217;ll look at Functions that process the blob and Write to CosmosDB</p>
</div>
</div>]]></description><link>https://dannylane.github.io/2017/07/10/Azure-Functions-with-Service-Bus-and-Blob-Storage.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/07/10/Azure-Functions-with-Service-Bus-and-Blob-Storage.html</guid><category><![CDATA[Azure]]></category><category><![CDATA[ Functions]]></category><category><![CDATA[ Serverless]]></category><category><![CDATA[ ServiceBus]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Mon, 10 Jul 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Preventing failing TFS builds from entering the continuous delivery pipeline - TFS 2013]]></title><description><![CDATA[<div class="paragraph">
<p><strong>TL;DR;</strong> Updating a TFS Build process to reorder test execution before automated release</p>
</div>
<div class="paragraph">
<p>I setup a continuous delivery pipeline for a client project I was working on recently and I  wanted to touch on a quirk I noticed after it was up in running.</p>
</div>
<div class="paragraph">
<p>At a high level the pipeline was developer check in → TFS 2013 Build → Run unit &amp; integration tests → Deploy to Test environment.
Once a release was deployed to the test environment the test team had the option of pushing the same build to higher environments SIT → UAT → PROD.</p>
</div>
<div class="paragraph">
<p>When I was going through the process with the test team about how to push the builds etc, one of them asked about builds that had failing unit tests, how would they know to avoid those builds.
I assured them that broken tests will fail the build so it won&#8217;t get released. I was wrong.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/1.png" alt="1.png">
</div>
<div class="title">Figure 1. I had expected this setting in the build definition to prevent builds with failing tests from being deployed.</div>
</div>
<div class="paragraph">
<p>After a few days I noticed a build in the test environment that had failed because of broken tests but somehow had still escaped into the wild.</p>
</div>
<div class="paragraph">
<p>After a little research I tracked the behavior down to the build process template.
One of the steps required to get you
r TFS 2013 builds integrated with MS Release Management (RM) is to configure the builds to use the release build template (ReleaseTfvcTemplate.12.xaml).
Using this process template adds the following config options to your build:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/2.png" alt="2.png">
</div>
</div>
<div class="paragraph">
<p>Setting the 'Release Build' flag indicates that you want the build to be released by RM.</p>
</div>
<div class="paragraph">
<p>It turns out that the sequence of events in the xaml file wasn&#8217;t what I had expected. In the screenshot below you can see that the "Process the release" step occurs before the "Run VS Test Runner" step.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/3.png" alt="3.png">
</div>
</div>
<div class="paragraph">
<p>I found some <a href="https://social.msdn.microsoft.com/Forums/vstudio/en-US/dcce6ce7-830c-4bc2-805b-1cf331e04253/release-management-releasing-failed-builds-also-triggering-release-from-failed-builds?forum=tfsbuild">others</a> had the same issue with TFS 2012 and the suggestions pointed me in the right direction but didn&#8217;t work on TFS 2013</p>
</div>
<div class="paragraph">
<p>The obvious first step is to change the order:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/4.png" alt="4.png">
</div>
</div>
<div class="paragraph">
<p>But changing the order on it&#8217;s own is&#8217;t enough.
I needed to update the "Process the release" step to only run if the build has not failed yet.</p>
</div>
<div class="paragraph">
<p>The BuildDetail properties tat were available in TFS 2012 need to be <a href="https://social.msdn.microsoft.com/Forums/vstudio/en-US/49f11ed9-9fa8-4c20-952a-d39ee7e71051/can-no-longer-user-builddetaildroplocation-for-copydirectory-with-tfs-2013-using-build-process?forum=tfsbuild">imported</a> when using TFS 2013</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/5.png" alt="5.png">
</div>
</div>
<div class="paragraph">
<p>With the BuildDetail variable available I can wrap the release step in a condition to that checks if the build has failed.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/failbuild/6.png" alt="6.png">
</div>
</div>
<div class="paragraph">
<p>Once you have updated the xaml, save it to your source control so it can be set as the new template for the build.</p>
</div>]]></description><link>https://dannylane.github.io/2017/04/03/Preventing-failing-TFS-builds-from-entering-the-continuous-delivery-pipeline-TFS-2013.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/04/03/Preventing-failing-TFS-builds-from-entering-the-continuous-delivery-pipeline-TFS-2013.html</guid><category><![CDATA[TFS]]></category><category><![CDATA[ UnitTest]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Mon, 03 Apr 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[An Amazon Alexa Skill Using AWS Lambda and the Strava API]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/alexa/header.png" alt="header.png">
</div>
</div>
<div class="paragraph">
<p><strong>TL;DR:</strong> I built an Amazon Alexa voice skill on AWS Lambda using the Strava API (and OAuth)</p>
</div>
<div class="paragraph">
<p>I got an Amazon Echo as a Christmas present (to myself&#8230;&#8203;) and have been really impressed with it. I expected it to be a geek toy that nobody else in the house would use but it gets used a lot by other family members for things like spotify and interactive games (my 4 year old likes to try and beat Alexa at 20 questions!). I think the reason that it has gained such traction in our home is down to the quality of the voice interaction.</p>
</div>
<div class="paragraph">
<p>I wanted to see how difficult it would be it write an app (Skill) for Alexa. I use <a href="http://strava.com">Strava</a> a lot so I decided to see how difficult it would be to get a Skill to consume the API. I&#8217;ve outlined the steps in setting up the Skill and some of the code used to query the API.</p>
</div>
<div class="paragraph">
<p>Before I get started, this is an example of the response from the skill:</p>
</div>
<iframe width="100%" height="166" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/314831609&amp;color=ff5500&amp;auto_play=false&amp;hide_related=false&amp;show_comments=true&amp;show_user=true&amp;show_reposts=false"></iframe>
<div class="paragraph">
<p><em>Disclaimer</em><br>
There&#8217;s going to be lots of screenshots <em>(boo)</em> of the AWS consoles for the first half of this post. The <code>code</code> <em>(yay)</em> is towards the end.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_aws_lambda">AWS Lambda</h4>
<div class="paragraph">
<p>The first step is to create a Lambda function that will host the endpoint for the Skill.</p>
</div>
<div class="paragraph">
<div class="title">There are some templates available but I just started with a blank function.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/1.Lambda.png" alt="1.Lambda.png"></span></p>
</div>
<div class="paragraph">
<div class="title">The second step allows you to select the triggers that will invoke your serverless function</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/2.Lambda.png" alt="2.Lambda.png"></span></p>
</div>
<div class="paragraph">
<div class="title">There are a number of predefined options available, choose the "Alexa Skills Kit" option</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/3.Lambda.png" alt="3.Lambda.png"></span></p>
</div>
<div class="paragraph">
<div class="title">Once a valid trigger is selected you can procede to the next step</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/4.Lambda.png" alt="4.Lambda.png"></span></p>
</div>
<div class="paragraph">
<div class="title">I chose the Java 8 Runtime but you can chose C#, Node, etc.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/5.Lambda.png" alt="5.Lambda.png"></span></p>
</div>
<div class="paragraph">
<div class="title">You need to specify what the handler of your skill will be, this is the Java method.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/LambdaConfig.png" alt="LambdaConfig.png"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="_skill_configuration">Skill Configuration</h4>
<div class="paragraph">
<p>Once the skill is set up and you have an <a href="http://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html">Amazon Resource Name (ARN)</a> to reference your Lambda function, you can set up your Skill to point at it.</p>
</div>
<div class="paragraph">
<p>You create a Skill from the <a href="http://developer.amazon.com">developer.amazon.com</a> portal</p>
</div>
<div class="paragraph">
<div class="title"><strong>Skill Information</strong> allows you to set the name of your skill and the name your users will use to invoke the skill (the invocation name)</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/SkillInformation.png" alt="SkillInformation.png"></span></p>
</div>
<div class="paragraph">
<div class="title"><strong>Interaction Model</strong> is where most of the configuration for how users will speak to your skillis defined.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/InteractionModel.png" alt="InteractionModel.png"></span></p>
</div>
<div class="paragraph">
<p>The interaction model is made up of <strong>Intents</strong>, <strong>Slots</strong> and <strong>Utterances</strong>.<br>
I defined 2 intents, one to allow a user to query their own activities and the other to query their friends activities.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
  "intents": [
    {
      "slots": [
        {
          "name": "ActivityType",
          "type": "LIST_OF_ACTIVITIES"
        }
      ],
      "intent": "WhoWasActive"
    },
    {
      "slots": [
        {
          "name": "Direction",
          "type": "LIST_OF_DIRECTIONS"
        },
        {
          "name": "Date",
          "type": "AMAZON.DATE"
        }
      ],
      "intent": "MyTraining"
    }
  ]
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Slots are like variables that Amazon extracts from the users request based on how the user invokes the intent.
There are built in slots like <code>AMAZON.DATE</code> and you can create custom slots too, <code>LIST_OF_ACTIVITIES</code> is a custom slot I defined.</p>
</div>
<div class="paragraph">
<p>The <strong>Sample Utterances</strong> are where you list the ways you expect users to interact with your skill. You associate utterances with Intents in this section aswell as adding your slots as tokens.
For example, if a user spoke <em>Who Ran</em> or <em>Who Cycyled</em> they would both match the utterance <code>WhoWasActive Who {ActivityType}</code>. The Intent that would be requested form your endpoint would be <code>WhoWasActive</code> and the <code>Slot</code> would contain an <code>ActivityType</code> of <em>Ran</em> or <em>Cycled</em>.</p>
</div>
<div class="paragraph">
<p>For more info see the <a href="https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/alexa-skills-kit-interaction-model-reference">docs</a></p>
</div>
<div class="paragraph">
<div class="title"><strong>Configuration</strong> The configuration step allows you to set your AWS Lambda ARN or else configure your own enpoint</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/Configuration.png" alt="Configuration.png"></span></p>
</div>
<div class="paragraph">
<div class="title"><strong>Account Linking</strong> The account linking is where you can set up an OAuth client token request.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/AccountLinking.png" alt="AccountLinking.png"></span></p>
</div>
<div class="paragraph">
<p>Making the skill work with the Strava OAuth took a while to configure correctly but didn&#8217;t require any code which was nice. Once this config is setup users get prompted to authorise the app from their Strava account
<span class="image"><img src="https://dannylane.github.io/images/alexa/stravaAuth.png" alt="stravaAuth.png"></span></p>
</div>
<div class="paragraph">
<p>Once the user authorizes the skill, Amazon takes care of requesting and refreshing the auth token. The token is supplied to your request handler with every invocation of the skill and you pass the token through to the API call. Painless.</p>
</div>
</div>
<div class="sect3">
<h4 id="_aws_eclipse_plugin">AWS Eclipse Plugin</h4>
<div class="paragraph">
<p>There is an AWS Plugin for Eclipse available <a href="http://docs.aws.amazon.com/toolkit-for-eclipse/v1/user-guide/setup-install.html">here</a>.</p>
</div>
<div class="paragraph">
<div class="title">The plugin exposes various AWS interfaces through Eclipse.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/1.eclipse.png" alt="1.eclipse.png"></span></p>
</div>
<div class="paragraph">
<div class="title">From the context menu of a project it is a simple wizard to upload a function to an existing Lambda.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/2.eclipse.png" alt="2.eclipse.png"></span></p>
</div>
<div class="paragraph">
<div class="title">Once associated with your account you can select which function to upload to</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/3.eclipse.png" alt="3.eclipse.png"></span></p>
</div>
<div class="paragraph">
<div class="title">Configuration settings can be updated before the deployment</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/4.eclipse.png" alt="4.eclipse.png"></span></p>
</div>
<div class="paragraph">
<div class="title">The plugin then packages up the project and pushes it to the S3 bucket that backs the Lambda function</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/5.eclipse.png" alt="5.eclipse.png"></span></p>
</div>
</div>
<div class="sect3">
<h4 id="_code">Code</h4>
<div class="paragraph">
<p>I used the <a href="https://github.com/amzn/alexa-skills-kit-java">alexa-skills-kit-java</a> library from Amazon for taking care of a lot of the boilerplate.</p>
</div>
<div class="paragraph">
<p>The entry point to the code is the <code>RequestStreamHandler</code>, this is the handler defined in the Skill config:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">public class StriderSpeechletRequestStreamHandler extends SpeechletRequestStreamHandler {
	private static final Set&lt;String&gt; supportedApplicationIds = new HashSet&lt;String&gt;();
	static {
		/*
		 * This Id can be found on https://developer.amazon.com/edw/home.html#/
		 * "Edit" the relevant Alexa Skill and put the relevant Application Ids
		 * in this Set.
		 */
		supportedApplicationIds.add("amzn1.ask.skill.&lt;snip&gt;");
	}

	public StriderSpeechletRequestStreamHandler() {
		super(new StriderSpeechlet(), supportedApplicationIds);
	}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>StriderSpeechlet implements Speechlet</code> class is where you handle events like <code>onLaunch</code>, <code>onIntent</code>, <code>onSessionEnded</code>, etc.</p>
</div>
<div class="listingblock">
<div class="title">This is a pretty quick and dirty onIntent handler, a production ready version could consider investing in getting Spring DI configured.</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">	@Override
	public SpeechletResponse onIntent(final IntentRequest request, final Session session) throws SpeechletException {
		IExecuteQuery query = null;
		Intent intent = request.getIntent();
		String intentName = (intent != null) ? intent.getName() : null;
		String intentDate = getSlotValue(intent, "Date");
		String intentDirection = getSlotValue(intent, "Direction");
		String token = session.getUser().getAccessToken();

		try {
			if ("WhoWasActive".equals(intentName)) {
				query = new FollowerQuery(token, intentDate);
			} else if ("MyTraining".equals(intentName)) {
				query = new SelfQuery(token, intentDirection, intentDate);
			} else if ("AMAZON.HelpIntent".equals(intentName)) {
				// TODO help me mario !!!
			} else {
				throw new SpeechletException("Invalid Intent");
			}
		} catch (Exception e) {
			throw new SpeechletException(e.getMessage());
		}

		SsmlOutputSpeech speech = new SsmlOutputSpeech();
		speech.setSsml(query.Execute().get(0));

		return SpeechletResponse.newTellResponse(speech);
	}</code></pre>
</div>
</div>
<div class="sect4">
<h5 id="_strava_queries">Strava Queries</h5>
<div class="paragraph">
<p>Strava exposes a RESTful API where a user, once authenticated, can query info about their activities, friends activities, segments, etc. The docs for the API are <a href="https://strava.github.io/api/">here</a>.</p>
</div>
<div class="paragraph">
<div class="title">Using Postman to test the API endpoint</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/postman.png" alt="postman.png"></span></p>
</div>
<div class="listingblock">
<div class="title">A stripped down version of the API response</div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
  "id": 321934,
  "athlete": {
    "id": 227615,
    "resource_state": 1
  },
  "name": "Evening Ride",
  "description": "the best ride ever",
  "distance": 4475.4,
  "moving_time": 1303,
  "elapsed_time": 1333,
  "total_elevation_gain": 154.5,
  "elev_high": 331.4,
  "elev_low": 276.1,
  "type": "Run",
  "start_date": "2012-12-13T03:43:19Z",
  "start_date_local": "2012-12-12T19:43:19Z",
  "start_latlng": [ 37.8, -122.27],
  "end_latlng": [37.8, -122.27],
  "photos": {},
  "map": {},
  "average_speed": 3.4,
  "max_speed": 4.514,
  "segment_efforts": [],
  "laps": []
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>These are the endpoints I was most interested in:
<code><a href="https://www.strava.com/api/v3/athlete/activities" class="bare">https://www.strava.com/api/v3/athlete/activities</a></code> - Logged in users activities
<code><a href="https://www.strava.com/api/v3/activities/following" class="bare">https://www.strava.com/api/v3/activities/following</a></code> - Friends activities
<code><a href="https://www.strava.com/api/v3/athletes/:id" class="bare">https://www.strava.com/api/v3/athletes/:id</a></code> - Used for backfilling the athlete name for friends activities</p>
</div>
<div class="paragraph">
<p>I used the <a href="https://github.com/dustedrob/JStrava">JStrava</a> library for querying the Strava REST API.</p>
</div>
<div class="listingblock">
<div class="title">Querying a users own activities, optional timestap and direction (before date or after date, as supported by the Strava API)</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">	public List&lt;String&gt; Execute() {
		List&lt;Activity&gt; activities;
		if ("before".equalsIgnoreCase(_direction)) {
			activities = strava.getCurrentAthleteActivitiesBeforeDate(_date.getMillis());
		} else if ("after".equalsIgnoreCase(_direction)) {
			activities = strava.getCurrentAthleteActivitiesAfterDate(_date.getMillis());
		} else {
			activities = strava.getCurrentAthleteActivities();
		}

		int max = 5;
		if (activities.size() &lt; 5) {
			max = activities.size();
		}
		return ParseResults(activities.subList(0, max));
	}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Querying a users friends activities</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">	public List&lt;String&gt; Execute() {
		List&lt;Activity&gt; activities = strava.getCurrentFriendsActivities();
		int max = 5;
		if (activities.size() &lt; 5) {
			max = activities.size();
		}
		return ParseResults(activities.subList(0, max));
	}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">I had to go back to the API to get the name of friends as the activity only includes the athlete ID</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">	private List&lt;Activity&gt; FillAthleteInfo(List&lt;Activity&gt; activities) {
		Map&lt;Integer, Athlete&gt; athletes = new HashMap&lt;Integer, Athlete&gt;();
		int currentId;
		Athlete athlete;

		for (Activity activity : activities) {
			currentId = activity.getAthlete().getId();
			if (!athletes.containsKey(currentId)) {
				athlete = strava.findAthlete(currentId);
				athletes.put(currentId, athlete);
			} else {
				athlete = athletes.get(currentId);
			}

			activity.setAthlete(athlete);
		}
		return activities;
	}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">Formatting the response for Alexa using SSML</div>
<div class="content">
<pre class="highlight"><code class="language-java" data-lang="java">	private String responseFormat = "%1$s %2$s &lt;say-as interpret-as=\"unit\"&gt;%3$.1fkm&lt;/say-as&gt; "
			+ "in &lt;say-as interpret-as=\"unit\"&gt;%4$dhours&lt;/say-as&gt;"
			+ "&lt;say-as interpret-as=\"unit\"&gt;%5$dminutes&lt;/say-as&gt;";

    private String FormatActivity(Activity activity) {
		double km = activity.getDistance() / 1000;
		int hours = activity.getElapsed_time() / 3600;
		int minutes = (activity.getElapsed_time() % 3600) / 60;

		return String.format(responseFormat, activity.getAthlete().toString(), activity.getType(), km, hours, minutes);
	}</code></pre>
</div>
</div>
<div class="paragraph">
<p><em>Yes, I was being lazy and inserted the type into the response directly, so run is spoken instead of the past tense.</em></p>
</div>
</div>
<div class="sect4">
<h5 id="_speech_synthesis_markup_language">Speech Synthesis Markup Language</h5>
<div class="paragraph">
<p>Alexa alows you to tailor how the response is delivered (spoken) to the user using <a href="https://developer.amazon.com/public/solutions/alexa/alexa-skills-kit/docs/speech-synthesis-markup-language-ssml-reference">SSML</a></p>
</div>
<div class="listingblock">
<div class="title">A response for one of my activities looks like this</div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">    "outputSpeech": {
      "type": "SSML",
      "ssml": "Danny Lane Run &lt;say-as interpret-as=\"unit\"&gt;16.2km&lt;/say-as&gt; in &lt;say-as interpret-as=\"unit\"&gt;1hours&lt;/say-as&gt;&lt;say-as interpret-as=\"unit\"&gt;8minutes&lt;/say-as&gt;"
    },</code></pre>
</div>
</div>
</div>
<div class="sect4">
<h5 id="_result">Result</h5>
<div class="paragraph">
<p>There code above is all very agricultural but it&#8217;s enough to get a response from Alexa. To test the Skill you can input an utterance from the test page and see the request that goes to your app (see the session token and how the slots/intent are broken up) and you can see the response also.</p>
</div>
<div class="paragraph">
<div class="title">To test the skill you can use the Service Simulator from the <strong>Test</strong> tab.</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/ServiceSimulator.png" alt="ServiceSimulator.png"></span></p>
</div>
<div class="paragraph">
<p>The request/response for the request to 'ask Strider <strong>Tell me my activities yesterday</strong>' is below:</p>
</div>
<div class="listingblock">
<div class="title"><em>Request</em></div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
  "session": {
    "sessionId": "SessionId.&lt;snip&gt;",
    "application": {
      "applicationId": "amzn1.ask.skill.&lt;snip&gt;"
    },
    "attributes": {},
    "user": {
      "userId": "amzn1.ask.account.&lt;snip&gt;",
      "accessToken": "&lt;snip&gt;"
    },
    "new": true
  },
  "request": {
    "type": "IntentRequest",
    "requestId": "EdwRequestId.&lt;snip&gt;",
    "locale": "en-GB",
    "timestamp": "2017-03-27T20:32:51Z",
    "intent": {
      "name": "MyTraining",
      "slots": {
        "Direction": {
          "name": "Direction",
          "value": "before"
        },
        "Date": {
          "name": "Date",
          "value": "2017-03-26"
        }
      }
    }
  },
  "version": "1.0"
}</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title"><em>Response</em></div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
  "version": "1.0",
  "response": {
    "outputSpeech": {
      "type": "SSML",
      "ssml": "Danny Lane Run &lt;say-as interpret-as=\"unit\"&gt;16.2km&lt;/say-as&gt; in &lt;say-as interpret-as=\"unit\"&gt;1hours&lt;/say-as&gt;&lt;say-as interpret-as=\"unit\"&gt;8minutes&lt;/say-as&gt;"
    },
    "shouldEndSession": true
  },
  "sessionAttributes": {}
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>The audio file that Alexa generates for the SSML above is the one I linked to at the top of the post:</p>
</div>
<iframe width="100%" height="166" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/314831609&amp;color=ff5500&amp;auto_play=false&amp;hide_related=false&amp;show_comments=true&amp;show_user=true&amp;show_reposts=false"></iframe>
<div class="paragraph">
<p><strong>Friend Query</strong></p>
</div>
<div class="listingblock">
<div class="title">A request for the utterance <strong>Who was active</strong> results in a different intent being delivered to the endpoint:</div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">    "intent": {
      "name": "WhoWasActive",
      "slots": {
        "ActivityType": {
          "name": "ActivityType"
        }
      }
    }</code></pre>
</div>
</div>
<div class="listingblock">
<div class="title">And the response is for one of the people I follow</div>
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">  "response": {
    "outputSpeech": {
      "type": "SSML",
      "ssml": "Brian McCarthy Ride &lt;say-as interpret-as=\"unit\"&gt;31.0km&lt;/say-as&gt; in &lt;say-as interpret-as=\"unit\"&gt;1hours&lt;/say-as&gt;&lt;say-as interpret-as=\"unit\"&gt;0minutes&lt;/say-as&gt;"
    },
    "shouldEndSession": true
  }</code></pre>
</div>
</div>
<div class="paragraph">
<p>The SSML for the response above sounds like this:</p>
</div>
<iframe width="100%" height="166" scrolling="no" frameborder="no" src="https://w.soundcloud.com/player/?url=https%3A//api.soundcloud.com/tracks/314831610&amp;color=ff5500&amp;auto_play=false&amp;hide_related=false&amp;show_comments=true&amp;show_user=true&amp;show_reposts=false"></iframe>
</div>
<div class="sect4">
<h5 id="_logging">Logging</h5>
<div class="paragraph">
<div class="title">Lambda <a href="http://docs.aws.amazon.com/lambda/latest/dg/java-logging.html">supports</a> Log4j and <code>LambdaLogger</code>, the logging output is available in AWS CloudWatch</div>
<p><span class="image"><img src="https://dannylane.github.io/images/alexa/CloudWatchLogs.png" alt="CloudWatchLogs.png"></span></p>
</div>
</div>
<div class="sect4">
<h5 id="_submission">Submission</h5>
<div class="paragraph">
<p>There is a rigerous approval process in place for Alexa Skills, I would need to improve a lot of areas of the code to make it robust enough to handle any level of testing so this may never see the light of day!</p>
</div>
<div class="paragraph">
<p>Thanks for reading.</p>
</div>
</div>
</div>]]></description><link>https://dannylane.github.io/2017/03/28/An-Amazon-Alexa-Skill-Using-AWS-Lambda-and-the-Strava-API.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/03/28/An-Amazon-Alexa-Skill-Using-AWS-Lambda-and-the-Strava-API.html</guid><category><![CDATA[AWS]]></category><category><![CDATA[ Alexa]]></category><category><![CDATA[ Lambda]]></category><category><![CDATA[ Strava]]></category><category><![CDATA[ API]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Tue, 28 Mar 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Troubleshooting TFS DB Growth]]></title><description><![CDATA[<div class="paragraph">
<p><strong>Spoiler:</strong> This was caused by ReleaseManager 2013 retaining builds. Read on for the work around that was put in place.</p>
</div>
<div class="paragraph">
<p>As part of a project I was working on recently; I put in a lot of work around TFS Builds, Unit/Integration Tests &amp; continous delivery using ReleaseManager.</p>
</div>
<div class="paragraph">
<p>It was a fairly standard pipeline:<br>
→ Check in → Unit &amp; Integration Tests Run → Build → Release To Test</p>
</div>
<div class="paragraph">
<p>Up until this project TFS Build hadn&#8217;t been used, TFS was just used for source control and some workitem tracking, it was hosted on a VM and space was never an issue.</p>
</div>
<div class="paragraph">
<p>After a few months of the project I started getting reports that the TFS DB size was growing constantly. The IT team had to keep allocating more space to the disks used for storing the DB backups.</p>
</div>
<div class="paragraph">
<p>It turns out that the <code>Tfs_DefaultCollection</code> had grown from about 6GB to over 50GB and the growth was constant.</p>
</div>
<div class="paragraph">
<p>After doing a little research online I found the following query that gives some insight into which tables are taking up the most space in the db.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-sql" data-lang="sql">USE Tfs_DefaultCollection

SELECT
    t.NAME AS TableName,
    s.Name AS SchemaName,
    p.rows AS RowCounts,
    SUM(a.total_pages) * 8 AS TotalSpaceKB,
    SUM(a.used_pages) * 8 AS UsedSpaceKB,
    (SUM(a.total_pages) - SUM(a.used_pages)) * 8 AS UnusedSpaceKB
FROM
    sys.tables t
INNER JOIN
    sys.indexes i ON t.OBJECT_ID = i.object_id
INNER JOIN
    sys.partitions p ON i.object_id = p.OBJECT_ID AND i.index_id = p.index_id
INNER JOIN
    sys.allocation_units a ON p.partition_id = a.container_id
LEFT OUTER JOIN
    sys.schemas s ON t.schema_id = s.schema_id
GROUP BY
    t.Name, s.Name, p.Rows
ORDER BY
    TotalSpaceKB desc</code></pre>
</div>
</div>
<table class="tableblock frame-all grid-all spread">
<caption class="title">Table 1. Tables taking up the most space</caption>
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<thead>
<tr>
<th class="tableblock halign-left valign-top">Table</th>
<th class="tableblock halign-left valign-top">Row Count</th>
<th class="tableblock halign-left valign-top">Size (GB)</th>
</tr>
</thead>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_TestResult</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">367386</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">19.660805</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_TestMessageLogEntry</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">22354468</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">12.085457</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_Content</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">62475</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">4.4598694</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_BuildInformation2</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">22736801</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">7.6754837</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Normally TFS retains builds as defined by a build retention policy, ours tells it to only keep the last 10 builds.<br>
However, there is a known issue with the Microsoft Release Management 2013 that causes every build that is involved in a release to be retained indefinitely, this doesn&#8217;t really fit with the continous delivery model and has been changed in newer versions.<br>
Every check in we do triggers a build which triggers a release <strong>so all our builds are retained</strong>.</p>
</div>
<div class="paragraph">
<p>This has issues for the storage capacity on the drops folder where builds are dropped and also storage capacity in the database.</p>
</div>
<div class="paragraph">
<p>The issue was exacerbated for this project because of the logging setup in the integration tests. Each test class library gets a DB deployed and dropped and seed scripts run, all the logging from these activities is echoed to the console. The application logging is also set to console output.<br>
All this log info for each test run is stored in the TFS database which grew unmanageably.<br>
The logging was turned off on the automated builds so this should help keep the size down for newer builds</p>
</div>
<div class="listingblock">
<div class="title">The application logs are driven by a config file but for tests I set the logs to <code>ConsoleOut</code> in <code>[SetUpFixture]</code> as the log information is available for each Unit Test. When I realised that all the logging info was retained in the DB I replaced the <code>ConsoleOut</code> logger with a <code>NoOp</code> logger on the build server.</div>
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">    public static class TestLogSetup
    {
        public static void Logger()
        {
            Console.WriteLine(Environment.MachineName);
            if (Environment.MachineName.Contains("&lt;ServerName&gt;"))
            {
                Console.WriteLine("No Op Logger");
                LogManager.Adapter = new Common.Logging.Simple.NoOpLoggerFactoryAdapter();
            }
            else
            {
                Console.WriteLine("Console Logger");
                LogManager.Adapter = new Common.Logging.Simple.ConsoleOutLoggerFactoryAdapter();
            }
        }
    }</code></pre>
</div>
</div>
<div class="paragraph">
<p>To manage the retained builds I wrote a powershell script that does the following steps for every build older than 2 weeks:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p>Update the Retain Indefinitely flag to false</p>
</li>
<li>
<p>Delete the build</p>
</li>
<li>
<p>Destroy the build</p>
</li>
</ol>
</div>
<div class="paragraph">
<p><strong>Delete the build</strong><br>
This is what is available through the UI, it is only a logical delete and while it does delete artifacts from the network drop folders, it does not delete rows from the db</p>
</div>
<div class="paragraph">
<p><strong>Destroy the build</strong><br>
This is the delete from the db. This is not available though the UI and is only available through the API</p>
</div>
<div class="listingblock">
<div class="title">This is the script that I wrote to destroy the builds.</div>
<div class="content">
<pre class="highlight"><code class="language-powershell" data-lang="powershell">param ($serverName = 'http://&lt;snip&gt;:8080/tfs/DefaultCollection')

[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.VersionControl.Client")
$tfs = [Microsoft.TeamFoundation.Client.TeamFoundationServerFactory]::GetServer($serverName)

$buildServer =  $tfs.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
$vcs =  $tfs.GetService([Microsoft.TeamFoundation.VersionControl.Client.VersionControlServer])

$projects = $vcs.GetAllTeamProjects($true) # We will run this for all team projects

foreach ($project in $projects){
    Write-Host $project.Name

    # Get all the build definitions for a given project
    $buildDefs = $buildServer.QueryBuildDefinitions($project.Name)

    foreach ($buildDef in $buildDefs){
        Write-Host "-" + $buildDef.Name
        # Delete all builds older than 14 days. This is a bit aggressive and can be pushed to a month once the logs are smaller on automated builds.
        $endDate = (Get-Date).AddDays(-14)
        # Keep going back until we hit dec 2014
        while($endDate.Year -ge 2014){
            $year = $endDate.Year
            $month = $endDate.Month
            write-host $year $month
            # Create a search object for the project/build def
            $buildDetailSpec = $buildServer.CreateBuildDetailSpec($project.Name, $buildDef.Name)
            $startDate = $endDate.addMonths(-1)
            $buildDetailSpec.MaxFinishTime = $endDate # Search criteria
            $buildDetailSpec.MinFinishTime = $startDate # Search criteria
            # only build info, not workitems, labels, etc
            $buildDetailSpec.InformationTypes = $null

            ## This is important for the query, builds that are deletd by retention
            ## or the ui are only logically deleted, not destroyed in the db
            $buildDetailSpec.QueryDeletedOption = [Microsoft.TeamFoundation.Build.Client.QueryDeletedOption]::IncludeDeleted

            # Search all the things
            $builds = $buildServer.QueryBuilds($buildDetailSpec)
            if($builds.Builds.Length -ge 1){
                Write-Host "before update" $builds.Builds
                foreach($build in $builds.Builds){
                    # Need to turn off the Keep Forever (Retain) flag set by Release Manager.
                    if ($build.KeepForever -eq $true){
                        Write-Host "Updating build " $build.Uri
                        # Gets an updatable ref to the build
                        $buildToEdit = $buildServer.GetBuild($build.Uri)
                        # Edit the build flag
                        $buildToEdit.KeepForever = $false;
                        # save the build back to the tfs
                        $buildServer.SaveBuilds(@($buildToEdit))
                    }
                }
                # refresh the query given that some of the builds have been updated
                $builds = $buildServer.QueryBuilds($buildDetailSpec)
                Write-Host "after refresh" $builds.Builds
                # Delete the build, test results, symbols, drop, etc.
                $buildServer.DeleteBuilds($builds.Builds,
                [Microsoft.TeamFoundation.Build.Client.DeleteOptions]::All)
                # Destroy the db records
                $buildServer.DestroyBuilds($builds.Builds)
            }
            # rolling back the years.
            $endDate = $endDate.addMonths(-1)
        }
    }
}</code></pre>
</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/buildpurge/dummyproject.png" alt="dummyproject.png">
</div>
<div class="title">Figure 1. I added the script to a dummy project in source control.</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/buildpurge/schedule.png" alt="schedule.png">
</div>
<div class="title">Figure 2. This project was set to run every morning at 3am</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/buildpurge/script.png" alt="script.png">
</div>
<div class="title">Figure 3. The powershell script is set to run after the build</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/buildpurge/logs.png" alt="logs.png">
</div>
<div class="title">Figure 4. The output from the script is available in the diagnostics tab of the build info from TFS.</div>
</div>
<div class="paragraph">
<p>The script could have been set to run as a scheduled task somewhere but I liked the idea of having it run within TFS as it made it easy for the dev team to monitor.</p>
</div>
<div class="paragraph">
<p>The impact of destroying the old builds was significant.</p>
</div>
<table class="tableblock frame-all grid-all spread">
<colgroup>
<col style="width: 33.3333%;">
<col style="width: 33.3333%;">
<col style="width: 33.3334%;">
</colgroup>
<tbody>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">Table</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Row Count</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">Size (GB)</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_Content</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">69733</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">1.889136</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_TestResult</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">174868</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.323192</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_LocalVersion</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">413449</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.122072</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_BuildCodeChange</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">148950</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.078016</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_Version</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">101484</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.068872</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_BuildInformation2</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">88673</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.064232</p></td>
</tr>
<tr>
<td class="tableblock halign-left valign-top"><p class="tableblock">tbl_Command</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">92974</p></td>
<td class="tableblock halign-left valign-top"><p class="tableblock">0.039192</p></td>
</tr>
</tbody>
</table>
<div class="paragraph">
<p>Tracking down the cause of the size increase was an interesting exercise, hopefully some of this detail may help someone else in future.</p>
</div>]]></description><link>https://dannylane.github.io/2017/03/22/Troubleshooting-TFS-DB-Growth.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/03/22/Troubleshooting-TFS-DB-Growth.html</guid><category><![CDATA[TFS]]></category><category><![CDATA[ ReleaseManager]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Wed, 22 Mar 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Getting Started With .NET Core Microservices Using Docker & VS Code on OSX]]></title><description><![CDATA[<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>I&#8217;ve started working on some .NET Core microservices stuff and thought I&#8217;d outline the steps I took to get a basic instance of a .NET Core service up and running in Docker on OSX.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_install_net_core">Install .NET Core</h4>
<div class="paragraph">
<p>The full instructions are <a href="https://www.microsoft.com/net/core#macos">here</a><br>
First, install openssl using <a href="https://brew.sh/">Homebrew</a></p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">brew update
brew install openssl
mkdir -p /usr/local/lib
ln -s /usr/local/opt/openssl/lib/libcrypto.1.0.0.dylib /usr/local/lib/
ln -s /usr/local/opt/openssl/lib/libssl.1.0.0.dylib /usr/local/lib/</code></pre>
</div>
</div>
<div class="paragraph">
<p>Install the .NET Core SDK from the page above.</p>
</div>
<div class="paragraph">
<p>I had an issue after the install where the dotnet command wasn&#8217;t exposed on my path. Found the following fix on one of the github issue <a href="https://github.com/dotnet/cli/issues/2544#issuecomment-220248063">comments</a>:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">ln -s /usr/local/share/dotnet/dotnet /usr/local/bin</code></pre>
</div>
</div>
<div class="paragraph">
<p>Some comments suggest it may be because I use <a href="http://ohmyz.sh/">oh-my-zsh</a>.</p>
</div>
<div class="paragraph">
<p>Once the sdk is setup successfully you can init a project:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">mkdir dotnet-microservice
cd dotnet-microservice
dotnet new
dotnet restore
dotnet run</code></pre>
</div>
</div>
<div class="paragraph">
<p>That should produce a simple <code>Hello World</code> output:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/helloworld.png" alt="helloworld.png">
</div>
</div>
</div>
<div class="sect3">
<h4 id="_install_vs_code">Install VS Code</h4>
<div class="paragraph">
<p>Now that we have the basic .NET Core app let&#8217;s install VS Code to use as our IDE. The OSX download is available at <a href="https://code.visualstudio.com/">code.visualstudio.com</a></p>
</div>
<div class="paragraph">
<p>We need to add some nuget packages to the project so the first step is adding the Nuget package manager.</p>
</div>
<div class="paragraph">
<p>Go to View &#8594; Extensions and search for Nuget, the Docker extension is also useful for adding docker commands to VS Code.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/nugetextension.png" alt="nugetextension.png">
</div>
</div>
<div class="paragraph">
<p>Once you have Nuget installed you can access Nuget commands in the command bar with F1:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/commandbar.png" alt="commandbar.png">
</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/KSubedi/net-core-project-manager/master/images/add.gif" alt="add.gif">
</div>
</div>
<div class="paragraph">
<p>I added Kestrel and a couple of MVC nuget packages, you can see the list in the <code>project.json</code> file further down.</p>
</div>
<div class="paragraph">
<p>Once you add a nuget package through the UI you will get prompted to restore (<code>dotnet restore</code> is the terminal command):</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/vscoderestore.png" alt="vscoderestore.png">
</div>
</div>
<div class="sect4">
<h5 id="_some_code">Some Code</h5>
<div class="paragraph">
<p>A microservice will normally expose a RESTful interface, to serve as the host we add a <code>WebHostBuilder</code> to the main method. Kestrel is a lightweight web server included in .NET Core. The <code>Startup</code> class is passed to the web host and allows you to define routing, DI and other services. The <code>ConfigurationBuilder</code> can be used to compile config settings from different sources; file, args, env, etc.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">        public static void Main(string[] args)
        {
            Console.WriteLine("Hello World!");

            var config = new ConfigurationBuilder()
            .Build();

            var builder = new WebHostBuilder()
                .UseContentRoot(Directory.GetCurrentDirectory())
                .UseConfiguration(config)
                .UseStartup&lt;Startup&gt;()
                .UseKestrel()
                .UseUrls("http://localhost:5000");

            var host = builder.Build();
            host.Run();
        }</code></pre>
</div>
</div>
<div class="paragraph">
<p>The Startup class here is pretty basic, I just went with the default routing provided out of the box by MVC. You can add different route patterns or you can use attribute based routing if desired.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">    public class Startup
    {
        public void ConfigureServices(IServiceCollection services)
        {
            services.AddMvcCore();
        }
        public void Configure(IApplicationBuilder app, ILoggerFactory loggerFactory)
        {
            app.UseMvcWithDefaultRoute();
        }
    }</code></pre>
</div>
</div>
<div class="paragraph">
<p>The controller is a simple math function. Um, multiplication&#8230;&#8203;</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">    public class CalculatorController : Controller
    {
        public string Multiply(int first, int second)
        {
            return (first * second).ToString();
        }
    }</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>project.json</code> lists the dependancies. In the most recent version of .NET Core, these <code>.json</code> files get upgraded to <code>.csproj</code> files</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-json" data-lang="json">{
  "version": "1.0.0-*",
  "buildOptions": {
    "debugType": "portable",
    "emitEntryPoint": true
  },
  "dependencies": {
    "Microsoft.AspNetCore.Server.Kestrel": "1.1.0",
    "Microsoft.AspNetCore.StaticFiles": "1.1.0",
    "Microsoft.AspNetCore.Routing": "1.1.0",
    "Microsoft.AspNetCore.Mvc.Core": "1.1.0",
    "Microsoft.AspNetCore.Mvc": "1.1.0"
  },
  "frameworks": {
    "netcoreapp1.1": {
      "dependencies": {
        "Microsoft.NETCore.App": {
          "type": "platform",
          "version": "1.1.0"
        }
      },
      "imports": "dnxcore50"
    }
  }
}</code></pre>
</div>
</div>
<div class="paragraph">
<p>Using <code>dotnet run</code> you can start the server:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/curl.png" alt="curl.png">
</div>
</div>
<div class="paragraph">
<p>The debug mode of VS Code lets you debug the application as you would expect:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/debug.png" alt="debug.png">
</div>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_docker">Docker</h4>
<div class="paragraph">
<p>Now that we can run the service locally, lets add it to a docker container. The first step is to create a <code>dockerfile</code>. The Docker extension for VS Code has some commands built in:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/dockercommands.png" alt="dockercommands.png">
</div>
</div>
<div class="paragraph">
<p>This was the first version of my <code>dockerfile</code>, I&#8217;ll walk through some of the issues I had to troubleshoot further down.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-dockerfile" data-lang="dockerfile">FROM microsoft/aspnetcore:1.0.1
LABEL Name=dotnet-microservice Version=0.0.1
ARG source=.
WORKDIR /app
EXPOSE 5000
COPY $source .
ENTRYPOINT dotnet dotnet-microservice.dll</code></pre>
</div>
</div>
<div class="paragraph">
<p>Building my docker file from VS Code:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">docker build -f Dockerfile -t dotnet-microservice:latest .
Sending build context to Docker daemon 518.1 kB
Step 1/7 : FROM microsoft/aspnetcore:1.0.1
1.0.1: Pulling from microsoft/aspnetcore</code></pre>
</div>
</div>
</div>
<div class="sect3">
<h4 id="_base_image_vs_app_version_mismatch">Base image vs app version mismatch</h4>
<div class="paragraph">
<p>Running the docker container:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">▶ docker run -it --rm dotnet-microservice:latest
The specified framework 'Microsoft.NETCore.App', version '1.1.0' was not found.
  - Check application dependencies and target a framework version installed at:
      /usr/share/dotnet/shared/Microsoft.NETCore.App
  - The following versions are installed:
      1.0.1
  - Alternatively, install the framework version '1.1.0'.</code></pre>
</div>
</div>
<div class="paragraph">
<p>My first issue was the incorrect version of my app compared to the aspnetcore version I pulled down in my base image.</p>
</div>
<div class="paragraph">
<p>I updated the base image in the dockerfile and rebuilt:
<code>FROM microsoft/aspnetcore:1.1.0</code></p>
</div>
</div>
<div class="sect3">
<h4 id="_copying_the_correct_folder">Copying the correct folder</h4>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">▶ docker run -it --rm dotnet-microservice:latest
Did you mean to run dotnet SDK commands? Please install dotnet SDK from:
  http://go.microsoft.com/fwlink/?LinkID=798306&amp;clcid=0x409</code></pre>
</div>
</div>
<div class="paragraph">
<p>The issue this time turned out to be the <code>COPY</code> command wasn&#8217;t copying my build output to the working dir of the image.</p>
</div>
<div class="paragraph">
<p><code>COPY $source/bin/Debug/netcoreapp1.1/ .</code></p>
</div>
</div>
<div class="sect3">
<h4 id="_publishing_artifacts">Publishing artifacts</h4>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">docker run -it --rm dotnet-microservice:latest
Error: assembly specified in the dependencies manifest was not found -- package:
'Microsoft.DotNet.PlatformAbstractions', version: '1.1.0', path: 'lib/netstandard
1.3/Microsoft.DotNet.PlatformAbstractions.dll'</code></pre>
</div>
</div>
<div class="paragraph">
<p>This time I realized I was copying the <code>build</code> output but I needed to be running a <code>publish</code> so that dependencies are included. I needed to run <code>dotnet publish</code> and change the <code>COPY</code> to pick up the published output.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">▶ dotnet publish
Publishing dotnet-microservice for .NETCoreApp,Version=v1.1
Project dotnet-microservice (.NETCoreApp,Version=v1.1) was previously compiled. S
kipping compilation.
publish: Published to /Users/danny/dev/dotnet-microservice/bin/Debug/netcoreapp1.
1/publish
Published 1/1 projects successfully</code></pre>
</div>
</div>
<div class="paragraph">
<p><code>COPY $source/bin/Debug/netcoreapp1.1/publish .</code></p>
</div>
</div>
<div class="sect3">
<h4 id="_port_forwarding">Port forwarding</h4>
<div class="paragraph">
<p>The <code>publish</code> did the trick, now I was able to run the docker container&#8230;&#8203; but not access the endpoint.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">▶ docker run -it --rm dotnet-microservice:latest
Hello World!
Hosting environment: Production
Content root path: /app
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.</code></pre>
</div>
</div>
<div class="paragraph">
<p>I ran the docker container specifying <code>*</code> as the host to bind to and forwarding port 5000 on the container to post 5001 on my machine.</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-bash" data-lang="bash">▶ docker run -p 5001:5000 dotnet-microservice
Hello World!
Hosting environment: Production
Content root path: /app
Now listening on: http://*:5000
Application started. Press Ctrl+C to shut down.</code></pre>
</div>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/curl2.png" alt="curl2.png">
</div>
</div>
<div class="paragraph">
<p>Curl confirms that there is nothing running on port 5000 and the container is exposed on port 5001.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/dotnetcore/dockerps.png" alt="dockerps.png">
</div>
</div>
<div class="paragraph">
<p>The <code>docker ps</code> command lists the running containers</p>
</div>
<div class="paragraph">
<p>That&#8217;s it, a simple microservice running on the .NET Core Kestrel server and packaged up in a docker image. It&#8217;s a basic structure for somethign that can be built upon in future.</p>
</div>
<div class="paragraph">
<p>I&#8217;ve included the terminal commands/output just in case others hit similar issues and hopefully the solutions above will help them.</p>
</div>
</div>]]></description><link>https://dannylane.github.io/2017/03/14/Getting-Started-With-NET-Core-Microservices-Using-Docker-VS-Code-on-OSX.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/03/14/Getting-Started-With-NET-Core-Microservices-Using-Docker-VS-Code-on-OSX.html</guid><category><![CDATA[.net]]></category><category><![CDATA[ core]]></category><category><![CDATA[ docker]]></category><category><![CDATA[ osx]]></category><category><![CDATA[ vscode]]></category><category><![CDATA[ microservice]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Tue, 14 Mar 2017 00:00:00 GMT</pubDate></item><item><title><![CDATA[Extending EntityFramework Generated Model Classes With T4 Templates]]></title><description><![CDATA[<div class="paragraph">
<p>I was working on a project recently where we used a common pattern for versioning DB records. Extending the EF generated model classes made it easier for the team to apply the pattern consistently.<br>
I updated the generated code to render a generic interface if specific properties exist on the model. 2 helper methods (<code>Clone</code> &amp; <code>Update</code>) were added to the models and were regenerated whenever the model was refreshed.</p>
</div>
<div id="img-project" class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/t4/project.png" alt="project.png">
</div>
<div class="title">Figure 1. EnttityVersion.ttinclude</div>
</div>
<div class="paragraph">
<p>I added new templating content as methods exposed in a <code>.ttinclude</code> file.</p>
</div>
<div id="img-t4" class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/t4/t4.png" alt="t4.png">
</div>
<div class="title">Figure 2. Add reference to .ttinclude file</div>
</div>
<div class="paragraph">
<p>Using DB first EF and generating the model from the existing schema uses a t4 template, <code>model1.tt</code> by default.</p>
</div>
<div class="paragraph">
<p>The <code>.ttinclude</code> file can be added to the top of this template and every model class that is generated will include the new content you define.</p>
</div>
<div class="listingblock">
<div class="title">RenderInterfaces template method</div>
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">&lt;#+void Render_Interfaces(System.Data.Entity.Core.Metadata.Edm.EntityTypeBase entity){
var members = ((EntityType)entity).DeclaredProperties;

var pk = entity.KeyMembers.Any(k=&gt;k.Name.EndsWith("SeqID"));
var hasEE = members.Any(m=&gt;m.Name=="EffectiveEndDate");
var hasES = members.Any(m=&gt;m.Name=="EffectiveStartDate");
var hasStatusID = members.Any(m=&gt;m.Name=="StatusID");

#&gt;
:IClonable&lt;&lt;#=entity.Name #&gt;&gt;&lt;#+if(hasEE &amp;&amp; hasES){#&gt;, IVersionable&lt;&lt;#=entity.Name #&gt;&gt;&lt;#+}#&gt;&lt;#+if(pk &amp;&amp; hasEE &amp;&amp; hasES &amp;&amp; hasStatusID){#&gt;, IStatusEntity&lt;#+}#&gt;
&lt;#+}#&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The <code>Render_Interfaces</code> method checks for specific columns/properties on the model and if they exist (<code>SeqID</code>, <code>EffectiveEndDate</code> &amp; <code>EffectiveStartDate</code>) then the generated class should have the interfaces <code>IVersionable&lt;T&gt;</code> &amp; <code>IClonable&lt;T&gt;</code></p>
</div>
<div class="listingblock">
<div class="title">Adding the <code>Render_Interfaces</code> method to the Model1.tt file</div>
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">&lt;#=codeStringGenerator.EntityClassOpening(entity)#&gt; &lt;#this.Render_Interfaces(entity);#&gt;</code></pre>
</div>
</div>
<div class="paragraph">
<p>The second method in the <code>.ttinclude</code> template is the one that adds the <code>Clone</code> and <code>Update</code> methods.</p>
</div>
<div class="listingblock">
<div class="title"><code>Clone</code> and <code>Update</code> method templates</div>
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">&lt;#+
void Render_Methods(System.Data.Entity.Core.Metadata.Edm.EntityTypeBase entity){

/// &lt;summary&gt;
/// Creates a shallow copy of the &lt;#=entity.Name #&gt;
/// Only copies primative types, should not copy primary keys
/// &lt;/summary&gt;
/// &lt;returns&gt;A new clone&lt;/returns&gt;
public &lt;#=entity.Name #&gt; Clone(){
	return new &lt;#= entity.Name #&gt;{
    &lt;#+
    for(var i = 0; i &lt; members.Count(); i++){
        if(!entity.KeyMembers.Any(k=&gt;k.Name == members[i].Name)){
		#&gt;
&lt;#=members[i].Name#&gt; = &lt;#=members[i].Name#&gt;,
	&lt;#+}
    }#&gt;
    };
}

/// &lt;summary&gt;
/// Updates the &lt;#=entity.Name #&gt; with values from the source
/// Only updates primative types, should not update primary keys
/// &lt;/summary&gt;
public void Update(&lt;#=entity.Name #&gt; source){
    &lt;#+for(var i = 0; i &lt; members.Count(); i++){
        if(!entity.KeyMembers.Any(k=&gt;k.Name == members[i].Name)){#&gt;
		this.&lt;#=members[i].Name#&gt; = source.&lt;#=members[i].Name#&gt;;
	&lt;#+}
    }#&gt;
}</code></pre>
</div>
</div>
<div id="img-t4" class="imageblock">
<div class="content">
<img src="https://dannylane.github.io/images/t4/render.png" alt="render.png">
</div>
<div class="title">Figure 3. Calling the <code>Render_Methods</code> helper method</div>
</div>
<div class="paragraph">
<p>The result of updating the interfaces and adding the <code>Clone</code> &amp; <code>Update</code> methods at the start of the class template looks like this:</p>
</div>
<div class="listingblock">
<div class="content">
<pre class="highlight"><code class="language-c#" data-lang="c#">public partial class Employee :IClonable&lt;Employee&gt;, IVersionable&lt;Employee&gt;{
/*
    Template debug info
    ---------------------
    Has PK ending with 'SeqID' : True
    Has EE : True
    Has ES : True
    Has Status : False
    */


/// &lt;summary&gt;
/// Creates a shallow copy of the Employee
    /// Only copies primative types, should not copy primary keys
    /// &lt;/summary&gt;
    /// &lt;returns&gt;A new clone&lt;/returns&gt;
    public Employee Clone(){
    	return new Employee{
        EID = EID,
    	PartyID = PartyID,
    	PreferredRoleID = PreferredRoleID,
    	AccessExpiryDate = AccessExpiryDate,
    	EffectiveStartDate = EffectiveStartDate,
    	EffectiveEndDate = EffectiveEndDate,
    	    };
    }

    /// &lt;summary&gt;
    /// Updates the Employee with values from the source
    /// Only updates primative types, should not update primary keys
    /// &lt;/summary&gt;
    public void Update(Employee source){
        this.EID = source.EID;
    	this.PartyID = source.PartyID;
    	this.PreferredRoleID = source.PreferredRoleID;
    	this.AccessExpiryDate = source.AccessExpiryDate;
    	this.EffectiveStartDate = source.EffectiveStartDate;
    	this.EffectiveEndDate = source.EffectiveEndDate;
    	}


    public int SeqId{
    	get{
    	return EmployeePartySeqID;
    	}
    	set{
    	EmployeePartySeqID = value;
    	}
    }</code></pre>
</div>
</div>
<div class="paragraph">
<p>Customizing the generated templates is pretty simple once you get the hang of the default template. I definitely prefer a code first approach, unfortunately that wasn&#8217;t an option on this project. While having to work with the <code>edmx</code> isn&#8217;t my preference, it is useful to know how to fine tune the models if needs be.</p>
</div>]]></description><link>https://dannylane.github.io/2017/03/07/Extending-Entity-Framework-Generated-Model-Classes-With-T4-Templates.html</link><guid isPermaLink="true">https://dannylane.github.io/2017/03/07/Extending-Entity-Framework-Generated-Model-Classes-With-T4-Templates.html</guid><category><![CDATA[C#]]></category><category><![CDATA[ T4]]></category><category><![CDATA[ EF]]></category><dc:creator><![CDATA[Danny Lane]]></dc:creator><pubDate>Tue, 07 Mar 2017 00:00:00 GMT</pubDate></item></channel></rss>