Release Management Using VSTS

If you have been tracking Azure at all lately you know that it is growing at breakneck pace. Microsoft has really turned up the volume on their enterprise cloud at all levels.  Just diving in can sometimes be a rough experience. Azure is a wide open field with many paths to solve your problem. I would like to show you the path I have found to get my release management up and running for my complex micro-modularity and microservices.

In the last post we created an ASP.NET Project for Visual Studio Team Services (VSTS) Release in a minimal way.  Now we will check ‘the check box of continuous integration’ and ‘push the button of continuous deployment’. Then we will add a second deployment environment to get your creative juices flowing. Continue reading “Release Management Using VSTS”

Release Management Using VSTS

ASP.NET Project for VSTS Release

So you have discovered the intense desire to manage your infrastructure as code and continuously deploy with your eyes closed. It is true, continuous integration and continuous deployment, once implemented, open a whole new set of opportunities. For me personally, most attractive of these is micro-functionality. Microservices are all the rage, but on a more fundamental level, it is hard to manage complex modularity without CI and CD.

For this reason I am going to assemble a refrence ASP.NET project that will demonstrate a common sinario in which the developer and Visual Studio are removed from the deployment process and endless deployments are made possible through Azure Resource Group template parameters. In a second post I will walk through setting up the Azure side of things. Continue reading “ASP.NET Project for VSTS Release”

ASP.NET Project for VSTS Release

Implementing IQueryable on a Business Logic Layer – Part 2

In Part 1 of this series I explained what sort of situation I am building towards.  In short I am building a highly modularized scaffolding system geared toward a microservice architecture.

Central to the functionality is a data object decorator and a gateway class that completely abstract away Entity Framework.  It was trivial to implement Take() and Skip(), passing the values to an internal IQueryable from EF.  Then I implemented ToList() to do a select as decorator.  As I mentioned in the previous post, that is great for doing linq against the gateway, but you cant cast it to a IQueryable as is expected in various places like WebApi. Continue reading “Implementing IQueryable on a Business Logic Layer – Part 2”

Implementing IQueryable on a Business Logic Layer – Part 2

Implementing IQueryable on a Business Logic Layer – Part 1

So you might be thinking this is a bad approach from the start.  That this sort of functionality belongs on the DAL or at least in a Repository.  However, in this age of Microservices and in the context of complex applications, Business logic will exist in several layers.  Think of it in MVVM or N-Tier style where there are complex validations and business logic that runs faster when closer to the DAL in a multi-tier environment. In this particular instance I am exposing this sort of module via oData and as an Actor via. Azure Service Fabric.

Getting the internals right is particularly important for me because I am using extensive code generation via T4. If I get it right, it quickly extends to my whole infrastructure of microservices.

Early on I thought I could implement just parts of IQueryable and be able to get away with it. I tried only implementing Where(), Skip() and Take() without the host of classes needed for an actual IQueryable.  This worked great when my code is loosely coupled to the implementation and blindly executes only these operations.

The catch is that I couldn’t just cast a partial implementation to IQueryable for things like WebApi to use. It would be great to just implement these three operations and have some sort of generic implementation of a decorator that bridges the implementation to an IQueryable. Alas, there is no such bridge in native .NET.  Thus, we must help ourselves.

Poking around the web you will find an ancient Microsoft secret Walkthrough: Creating an IQueryable LINQ Provider.  Many posts about this subject fail to address the fact that you may not be using any sort of IQueryable under the hood. The MSDN post shows you how without directly addressing it directly.

At a high level: during the Execute() phase you will need to figure out what you can pass on, do so, and then execute the underlying query to return your list.  This list then becomes the subject of the remainder of the query.

The following post will walk through my implementation thought process.

Implementing IQueryable on a Business Logic Layer – Part 1

T4 Templates in Visual Stuido

There are not many places on the web to understand T4.  As I have been poking around T4 lately here are a few notes I have gathered together.  One thing of interest is Razor Generator I have talked about before.  It can be used as a T4-like preprocessor template although I feel like it is less mature. Continue reading “T4 Templates in Visual Stuido”

T4 Templates in Visual Stuido

Sticking to Default Routs in ASP .NET MVC

When dealing with the sharing of code on a project, I feel like having conventions and standards and following are of the utmost importance.  Only by following the patterns can someone else walk in and understand what is going on.  It also helps newer programmers by letting their acquired knowledge apply across environments.

Occasionally I see people really get into routing when dealing with a web MVC framework.  I have even seen some who place custom routing in their top 5 must haves when looking for a web MVC framework. While it can be fun and rewarding, the most elegant solution sometimes is the one that is default.  Further in my environment we hold to the standard that the simplest is usually the best.  This means using default routing if possible, period.  The first choice we made was already MVC .NET and therefore adhearing to the internal conventions is implied.  For the sake of this conversation we will be assuming we are using MVC as a UI.  API functionality beyond REST may or may not require a different approach. Let us take a walk down the path of :

‘What does that mean practically?’

Here is the default route registered in a new project:

Default Route

Imagine you publish to a place that is reachable via http://example.com/.  Say you have a controller called MyController that resides within the Controllers folder in your project.  It has two methods : Index() and World().  If you go to http://example.com/My you will see the output of the Index() action on MyController since the default action is Index().  You can also execute the same action by going to http://example.com/My/Index.  Hopefully this aids in making sense of the fact that if you visit http://example.com/My/World the action MyController.World() is executed.  Also, by understanding how defaults work, you understand that when you visit http://example.com/ the routing is wanting you to have a controller named Home with an action method of Index().

This brings us to the first point of contention I will acknowledge. That of the third parameter ‘id’.  If we were going to leave the default routing and simply use this value, we end up having method parameters named id when they may be used as something like ‘name’.  Here is where my team and I have made the call that if it is any sort of identifier (like name or GUID, which it usually is) that we leave the default routing and reassign the value to a more specific variable inside the method.

Note that this routing pattern is matching the URL and not any parameter values like those sent in a HttpPost.  For example the Login() method of the default AccountController takes a LoginModel and a returnUrl without changing the default route.  The LoginModel is set via POST values and often the returnUrl is added to the URL queryString like http://exmaple.com/Account/?returnUrl=some_value_here.  On my team these are the preferred methods of assigning values when building with .NET MVC.

Does this mean we never add routs.  The short answer is: We never add routs.  We may make exceptions in extreme situations, but we have not come across any that need it. This quickly brings us to Areas.  Yes we use the defaults here too.

Areas

Creating an Area by the name of AreaName you are given the routing:

Default Area Route

So in this case if you had a controller named MyController like mentioned above but in the project folder Areas\AreaName\Controllers you could get to the World() action method by visiting http://example.com/AreaName/My/World.  Because we often build our controllers for areas in external libraries we add a little salt to our area routs. *Gasp* However, we still don’t actually modify the default route.

Area Route Modified

As you can see above, we add the namespace for the area and a default controller.   This is still the only routing we need.

 Conclusion.

You still may be asking yourself, where does this rule stop? The answer is simple: When making things work with the default routs is more complicated than route clutter.  I have yet to see where this is the case and it doesn’t warrant a whole new project.  Keeping it simple is an art.

Sticking to Default Routs in ASP .NET MVC

MVC Area in an External Assembly

The majority of ASP .NET MVC sites I build I treat the site itself as a single UI.  I don’t bother breaking it down into pieces, it is an MVC after all.  Thus it already addresses separation of concerns.  But what happens when your site gets large and you want to break it into application verticals within one MVC host.  The first thing you do is break the site into Areas.  This makes it possible to keep things organized.  Now lets take it a step further, and you would like your area to be in a separate assembly.  Maybe it is shared between applications, or you just want to track the development and control deployment a bit more.

I Googled all over trying to find a good solution.  The most helpful post came from Patrick Boudreaux a couple years ago.  Working your way through Patrick’s post you see that Javascript and Css still an issue:

To make a fully self-contained portable area, static content such as css, images and javascript needs to be embedded in the assembly, and found in a special way, just as the view templates are.

Thus we will include a way to solve that issue in my own ‘sepcial way’.  Mileage may vary, so let me know in the comments.

You can find the example code on GitHub. This series will be known as The Poor Mans Modular Area for ASP .NET MVC or PMMA for short.
works on my machineWe will try to make this as simple as possible.  I feel like ‘Simple‘ is a forgotten art form in the .NET world.  It would be great to just include a reference to another MVC application and use it’s areas.  If you try that though, you will find issues with both projects trying to take the wheel to drive.  We can avoid that by simply removing redundant pieces of the hosted Area.  Also, I want to simply and clearly prove the way the application is running, so we will assemble a prof of concept project.  We will  perform a simple string assignment in the Area’s controller, print that string in a View and load a Javascript file and a CSS file that all live in the Included project.

Add Area

The Project

Starting the project created a new MVC 5 solution and called it ‘MasterUI’.  Then add a second MVC 5 project called ‘Included’.  Next add an area by right clicking the Include project and selected Add > Area and name it “TestArea”.  Then within the area right clicked on the Controller folder and added a controller named ExampleController.cs.  Then inside the Index action add the line:

ViewBag.Message = "SUCCESS!!!";

Now right click on the Index() method name and selected add View.  My view Code looks like this:

@{
    ViewBag.Title = "Index";
}

@Styles.Render("~/Included/Content")

<h1>EXAMPLE AREA</h1>
<h2>@ViewBag.Message</h2>

@section scripts
{
    @Scripts.Render("~/Included/Scripts")
}

You will notice the Styles.Render() under the default ‘Title’ assignment that refer to a package with a URL that includes the Area name.  I did this to keep convention and keep it clear where the script is coming from.  It may make sense to be even more explicit, but this will be OK for this context.  Then note it will place the message inside H2 tags to prove the controller is talking to the view.  The Scripts.Render() call will place some scripts in the ‘scripts’ layout section of the master layout.  It too refers to the Area name.

From the Included project delete Global.asax, Controllers, fonts, Models and Views. In the ‘MasterUI’ project add a reference to the ‘Included’ project.  Run MasterUI and navigate to /TestArea/Example and prove you get a 404.

The Magic

Custom ToolThe first thing you want to do is add a Visual Studio extension that allows you to precompile your MVC Razor views.  Open Extensions and Updates and search for and install Razor Generator.  Razor Generator has two pieces: a VS extension and a NuGet package.  These enable you to compile your Razor Views into the library.  Enabling this is a little manual, but the payoff is well worth the effort.  Once you install the VS Extension, manage the NuGet packages on the Included project, search for and add RazorGenerator.Mvc.  As long as we are thinking about our views, browse to TestArea\Views in Solution Explorer.  Locate the _ViewStart.cshtml and view it’s properties.  Here is where the manual part happens.  You want to type “RazorGenerator” into the ‘Custom Tool’ field.  This should cause a .cs file to be generated from your view.  There is no need to edit the .cs file, simply make your changes in the .cshtml and it will be updated for you.  Next do the same to Example\Index.cshtml.

Now it is time to setup our static content.  ASP .NET MVC has great static content handling built-in.  We will leverage this by using a custom bundle transform.  You may not consider my method to be “correct” but it seems to work.  I am open to suggestions, let me know in the comments.  The class implements IBundleTransform as follows:

public class StringResourceTransform : IBundleTransform
{
    private List<String> _resourceFiles = new List<String>();
    private string _contentType;

    public StringResourceTransform(List<String> resourceFiles, String contentType = "text/javascript")
    {
        _resourceFiles = resourceFiles;
        _contentType = contentType;
    }

    public void Process(BundleContext context, BundleResponse response)
    {
        string result = String.Empty;

        foreach (var resource in _resourceFiles)
        {
            using (Stream stream = Assembly.GetExecutingAssembly()
                .GetManifestResourceStream(resource))
            {
                using (StreamReader reader = new StreamReader(stream))
                {
                    result += reader.ReadToEnd();
                }
            }
        }

        response.ContentType = _contentType;
        response.Content = result;
    }
}

This allows you to specify a Bundle by giving a list of Resource paths.  When the request comes in, the resources are read into a string and passed back out to via the response.  Given the above class, you would then specify your bundles in the Included project’s Bundle config similar to the this:

var scriptsBundle = new Bundle("~/Included/Scripts", 
  new StringResourceTransform(new List<String>() 
    { 
      "Included.Scripts.test1.js" 
    }));
scriptsBundle.Transforms.Add(new JsMinify());
bundles.Add(scriptsBundle);

var cssBundle = new Bundle("~/Included/Content", 
  new StringResourceTransform(new List<String>()
    { 
      "Included.Content.Test.css" 
    }, "text/css"));
cssBundle.Transforms.Add(new CssMinify());
bundles.Add(cssBundle);

Above first specifies a JS file that is located in the Included project under the Scipts folder by the name of test1.js.  You notice that the contentType defaults to “text/javascript” so it doesn’t need to specify this on these sorts of files.  The second set of lines do the same for a css file located in the Content folder.  The path is specified as the location within the project by setting the files to be an Embedded Resource.

First place a file in the Scripts directory by the name of test1.js.  Place the following text into it:

alert('success!!');

Then in its properties set ‘Build Action‘ to “Embedded Resource”.  Next, add a file by the name of Test.css to the Content directory and also set its Build Action to “Embedded Resource”.  Add the following text to it.

h2
{
    color: red;
}

Now head back to the MasterUI.  There are only a couple changes that need to be made to make it aware of Included.  With all the work we did in the Bundle config, simply add the following to the end of the BundleConfig in MasterUI:

Included.BundleConfig.RegisterBundles(bundles);

This will ensure the bundles fire.  Now you just need to make sure the routing can find the controllers.  The following line could be placed several places, but I just put mine in the Global.asax:

ControllerBuilder.Current.DefaultNamespaces.Add("Included.Areas.TestArea.Controllers");

Success

This bit of magic lets the routing know to look in our Included project area namespace and find controllers.  This way you don’t have to make any crazy changes to routing or build a custom anything else. Now run the application and go to /TestArea/Example.  If you see an alert that says ‘success!!” and a result like the screenshot you did it right.

Enhancements

As you may be thinking, you could certainly do this without having a sub-folder ‘Areas’.  The reason I did this is to make it clear we are dealing with an area, and to keep the namespace a little more organized.

If you are running in debug mode optimizations are off so you have to enable them with

BundleTable.EnableOptimizations = true;

But the problem then is that you have all your minification turned on too.  This is addressed by simply surrounding your minify transforms with #if(!DEBUG) compiler directives.

Conclusion

Using this method I plan on creating a multi-facet application that will be a one stop shop for custom tools where I work.  I will keep you posted on how it works out for me, feel free to do the same or ask questions in the comments.

MVC Area in an External Assembly

Schedule with Dante and PHP

Many PHP programmers are comfortable with the Unix/Linux server world.  While PHP has certainly flourished as part of a LAMP stack, one must not forget it’s portability.  PHP melds well with a Unix tool chain of pipes.  Something that is often over looked is how it melds with a Windows environment.  What if you want to express your creativity with Internet applications, but find yourself working in a entirely Microsoft environment?  PHP should find its way high on your list of technologies.  What if you are already using PHP and have access to Windows servers?  You should be taking advantage in their hidden potential.

One of the features of the windows operating system that deserves exploiting is Task Scheduler 2.  Task Scheduler 2 was introduced on Windows Vista and Windows Server 2008.  At first glance you may just see Cron with a GUI.  It isn’t like they dedicated a Microsoft Certification to it and you would be hard pressed to find Task Scheduler training.  However Task Scheduler 2 is much more than run Foo at the given date and time.  You have a wide variety of triggers, conditions and options surrounding the execution of the given command.  Time being one trigger of many including immediate queuing.   This single ‘hidden’ potential could be reason enough to have a single Windows Server in your environment.  Fortunately for us Task Scheduler 2 comes with all versions of Windows desktop and server.  Licensing often being the pivot point in many IT organizations, Windows Webserver 2008 is a modest price and does come with the two important pieces we will be taking a look at now.  That is: Task Scheduler 2 and IIS.

From a programmers perspective, using a piece of technology as different as Task Scheduler and COM may seem to require more effort than they are worth.  Here is where I would like to introduce Dante Job Scheduler.  A project in development, Dante is meant as the broker to your Windows Server’s job scheduling and queuing abilities.  Dante is built on the Crossley Framework which contains a growing number of classes that wrap COM and .NET functionality.  You can easily take advantage of this  in your application.  For now, I would like to show you the early functionality built into Dante’s UI and Rest style application services.

To get started, download Dante, and uncompressed it on your computer running Windows  in a place you can point an IIS site at.  The directory structure is simple, but the one we will be concerned with first is called ‘public’.  Open Internet Information Services Manager and create a new site.  Use the path to the public directory as your value for the ‘Physical path’.

When you point your browser at the site, you will be prompted for a user name and password.  The easiest one to use is one for a known profile on the box. Administrator can work fine but you may want to consider creating a dedicated account for Dante.  If you are working on a development machine it would be great for the sake of this demo if you just logged in as the user you are logged in as now.  This will allow you to see things in action.

The first thing you see is a list of scheduled tasks that may already be there for that profile.  If you logged in with a regular user account you may see things like Google Update listed.  To create a new task click the Create tab at the top of the page.  In this form, you should give it a recognizable unique name.  Then in the ‘Command’ field, enter the path to notepad.exe (c:\Windows\notepad.exe worked for me).  Enter a run date of the present date, and then a time in the near future.  Click submit and then you can view it under the ‘List’ tab.

While you are waiting for it to run (it may not show up the second it is scheduled, it may need a second to show the GUI), click on some of the tasks to see their details.  You may notice the XML link at the bottom of the details list.  This is the XML definition of the task.  An application can create such an XML and pass it to the scheduler as a task envelop.  The ‘Run’ and ‘Stop’ links do just as you would assume: run and stop a executing task.

You can experiment on the task you just created.  Click the details of the task and click ‘Run’ to  see the notepad GUI appear.  Then Click the ‘Stop’ link to see it close.  Not overly exciting unless you understand the potential.  If you have Firebug running you may have noticed the XHR calls being made via jQuery.  Each of these can be imitated from a remote computer.  Say you want to run a task called “MyTest”  you would simply make a request for https://<username&gt;:<password>@<server_address>/tasks/run/MyTest.  The same can be said of creating a new task.  To create a task simply form a request with your favorite HTTP client to: https://<username&gt;:<password>@<server_address>/tasks/new/?tskname=<taskname>&cmd=<command>&rundate=<date>&runtime=<time>.

As mentioned before there are many triggers and options available in Task Scheduler 2.  Most of which will find their way into Dante and Crossley Framework.  Soon a client class will also be available to further simplify your development.  Exposing COM functionality with PHP takes some patience.  Even so, if you are running PHP on Windows, great benefits can be gleaned.

I hope this taste has peaked your interest.  If you are so motivated I encourage you to experiment with Dante and share your thoughts or even code.

Schedule with Dante and PHP

Code Generation

Code Generation has long been an interest of mine.  No matter if it is PHP, C# or now VB.NET code generation can gets me up to speed with the basics fast.  RAD is something that left a bad taste in my mouth.  In fact most anyone who actually supports their work cringes when you say RAD or mention code generation.  With the Framework craze these days though, there are many places where code isn’t generated, but the same problems are encountered with the convention over configuration crowd.  So I approach code generation with caution.  The sad fact is that there are things that we repeat over and over again.

How many times have you typed “SECT * FROM foo”?  The key is keeping code generation extensible.  Where technology in general falls apart is where it tries to think for you and have it’s own preference where preference is the only deciding factor.  Frameworks that are good but require you to conform to their school  of thought only find acceptance amongst those who already think that way or don’t have enough experience under their belt to have an opinion yet.  The more generic and customizable the better.

Enter T4, the new code template system in Visual Studio 2008.  This is a great technology. As I learn it I find myself weaving more repeditive code, but with better quality.  It makes CRUD controllers a dream to deal with, and holds the potential for saving alot of time.  As I come across usefull T4 resources I will post them here.

I am interested to see how this changes my perception of my own code generation.  The T4 template usage is interesting.  They really allow for alot of logic within the template makign complicated generation possible.  Already I see this making it’s way into my own generator.

Code Generation

OOD In an MVC Land

If you are familiar with asp you are probably familiar with the concept of code behind and more recently MVC. Over the years I have bounced around using many different languages, frameworks and technologies. Using the pattern in another language emphasizes that this is best implemented as a conceptual pattern. OOD is best done first if you have any logic at all. If you are just editing a list and it IS just a CRUD app, an MVC up front approach might work fine. Every where I look the MVC is pushed as THE solution. When, like any other design pattern, it is never more than a piece. Try to think outside the MVC box and do what works. Don’t try to cram everything you want to do into the MVC or you will then need to create classes called ‘helpers’ and then the only business object you really have is the MVC, and you might as well have not done OOP in the first place, and you should have stuck with Clasic ASP or PHP4 and your procedural, monolithic methodologies.

Remember to keep your business logic out of your display logic. If you call it a template or a view is up to you. Just don’t be caught with your pants down when you find out your model is hard-coded to your business logic. Or worse, you find application has become a big string of classes that are untestable and codependent on your control structure. I always keep focused on being loosely coupled and dynamic without being magic.
Which brings me to the ASP MVC.NET implementation. As with many things in .NET, it seems to be a situation of ‘we want that buzzword tech. too!’. There are many advantages, and as usual I get the feeling it is a step in the right direction. Also not a surprise, it feels like it is not what is supposed to be. Like the point was lost in the translation. The .NET crowd wanted Ajax, they got the word but missed the point. their websites were freed from having to load the whole page, but they still were cursed with throning the state string back and forth and never got the benefit of good efficient client side processing with JavaScript.
Oh man, I don’t want to get started on JavaScript in the MS camp. Almost all the MS shops I have talked to in the last year think ‘Java’ and ‘JavaScript’ are terms that are interchangeable. To their credit, for all they knew of either they might as well be.
Then the .NET crews whined about not having flash, so they got Silverlight. (which I like what I have seen, but why cant every one just love SVG???)
My largest point of contention with ASP.NET is that all the training and documentaiton pushes you to use the built in controls. The built in controls are obviously not meant for the Internet, but are for LAN apps. Thus ASP.NET is best suited for application developers moving into the Web Interface realm. I once saw a ASP.NET application that fed 6MB HTML (not including graphics) at the user. This was the worst case, but was far from atypical.
Enter the ASP MVC. None of the state dependent controls work. Pat them on the back for destroying the only thing the rest of the web development world envies them for. Wait, this isn’t a totally negative rant. ASP MVC encourages you to be conscious and intentional with your HTML. It also opens the door for using jQuery instead of Microsoft Ajax. Thus, as far as the client/browser side you are free to do as you please.
But, then you may think that all is well. Until you start dealing with the server side. I am not going to get down on it, since it is still in development. Although, as with any software if it is out of development it is dead, so take that as you will. The real problem that I do see is that it tries too hard to be an MVC. Your Controllers end up getting huge with ‘actions’ (and where is that /really/ in the pattern). There is only one layer of organization in the controller. This means you cant go beyond controller/action. Granted you can customize your routing, which feels like you are configuring someone else’s app, instead of building your own. Then all your views tend to be one per action which defeats at least the bulk of the benefit of application and presentation separation.
Hear again, another MS team tries to step in and save the day. The Entity Framework (EF) is the official flavor of the day. This essentially is your ORM Model to supply your CRUD. None of which lend themselves to good OOD. You still are encouraged to draw up a action that feels oddly procedural mashing your EF object lists into a view. EF does not allow for any type of static extension so there is no natural way to build business objects. The solution to this is to just build your business objects facade over the EF ORM. But no examples encourage this, neither does any of the community publicize this. So you should shove your monolithic processing in an action and pray you can rewrite instead of maintain. Now to add to the sadness of it all, there is no straight forward way to call and render an action from another controller. So now that you have a huge controller class you now start copy and pasting code between actions. I have seen Quick Basic apps that are better.
Maybe I am just jumping the gun. It is still in development after all. Maybe the ASP MVC community will get there.
Currently the way I am dealing with everything is having a nice thick jQuery driven client. I load partial views in via Ajax and call JSON services. No community so far has dogged the MVC fail bullet. I just want to know when everyone is going to move on from MVC and realize OOP != OOD

OOD In an MVC Land