After watching a recent video containing Bill, I may just have to rethink my opinion of him. Watch the video below, it is from some US talk show where he, based on his own self-admitting ignorance, arrogantly supports the SOPA and labels any anti-SOPA components in the community or government as thiefs.

Luckily, the other speakers on the panel seemed to have done their homework and not just read the SOPA coversheet.  Now I wonder if he was just as arrogant for his Religulous documentary?

No doubt if SOPA goes through this article will be in violation to the act and the entire WORDPRESS site will be shutdown because of me.   Hmmm

Take a look for yourself:

Highly recommended.

This is most likely a good thing as Microsoft’s venture is slightly unproven whereas Hadoop has many many contributors (particularly Yahoo and Cloudera) and has been around for quite some time now.

Read the full article here

Read the full article here

http://www.wired.com/gadgetlab/2011/11/adobe-kills-mobile-flash/ [1]

In this day and age of being told what we should think, whether if we should like a film or not; a restaurant or not;  a make of car or not;  or what type of food we should minimise; so to is the almost equivalent to religious evangelism publicised by certain authors, speakers or even bloggers (yes I am a blogger too) on their idea on how we should program,  or SDLC patterns such as source code control or how to implement say continuous integration and/or deployment.   Many of the suggestions are brilliant, helpful and well deserved, particularly for those with little experience in the field.   However it is all too easy to become a disciple of the new religion without questioning what is being said.

As with most things,  a teacher with incorrect information or a teacher with an agenda is a bad teacher with a motive.  The same is said for programming authors or speakers.  Perhaps they want to sell you more books?  Have YOU thought about it?

I mentioned that many will accept an idea without questioning the mechanics behind it – take database object relational mapping (ORM) made arguably famous by a certain well known author and speaker.  People like it cause it makes the DB look like an object-orientated tree graph.  Nothing wrong with that, I particularly like canonical protocols or singular agnostic APIs.   If I were to take the critical approach to ORM I could encounter:

“awww everyone else is doing it;  It’s much easier!”  - they say

Is a common defence.   Fair enough, but it demonstrates the obvious and unfortunate scenario that certain developers will do a thing if there is sufficient evidence for doing so in the first place (particularly if it requires zero-coding on their part) or if it makes their job easier.  The alternative is not to use ORM but to use more complex and time consuming patterns such as say stored procedures.  I’m not saying stored procs are better as that would be hypocritical to this topic.  I’ll let you decide.  I find some developers are not aware or do not want to think about and or investigate for themselves the overhead of data transformation let alone network activity.

Which is faster -

• the system where the majority of data processing happens inside the database in stored procs with minimal data transformation or
• the system which the majority of data processing occurs external to the database,  subject to data transformation via an ORM data typically on another computer via a network?

I’ll let you decide that.

It doesn’t have to be an ORM either.  Perhaps they have not fully thought about log4net, StructureMap, ASP models.  Are they good tools?  What are the pros?  Are there any cons?  A project for the reader.

“Ease does not necessarily imply high performance in arguably the same way as complexity does not imply unreliability.” – Me

Many say simplicity is best.  Perhaps.    The question is have YOU thought about it?  What does it mean to you?  A project for the reader.

Which is better – a turn of the century zepplin or a contempory Boeing 747 Jumbo Jet?  Which is more reliable?  Which has more fail-safes?  A project for the reader.

Don’t let your instincts prejudice your eventual decision making process.  Think before you say something I always say.  Question and debate those in authority, but try not to disagree if the only evidence on your behalf is that others are doing it your way.  You have not contributed anything personal to the debate.  Again think for yourself and come up with your own evidence.  This is what I encourage of my juniors.

“Extraordinary claims requires extraordinary evidence”

Some programmers sadly are so keen to use well known off the shelf systems with little thought as to their actions.

e.g.

A need for inversion of control (IoC) such that a database or logging object can be wired to client objects is quite understandable.  Such things are quite nice and do seem to make our jobs easier.  IoC systems typically require configuration as I have blogged before.  You must ask yourself -

• is the configuration difficult?
• Is the configuration error prone?
• Is the configuration open to security vulnerabilities?
• Does it make it all worthwhile?
• Have you considered a plain 1990s Class Factory Pattern?

A project for the reader.  Again ease may not necessarily mean correctness.  What do you think?

Don’t think something just because someone told you to – learn to think for yourself again.

Again, this is not specific purely in the IT field but can apply to our daily lives too.

At the time of writing, ASP.NET development requires that all pages (for pre MVC), views, controllers and other content be in the web application project itself, which is a tad silly when you think about it. Some clever chaps (e.g. Portable Areas) have been working on ways to move the content to separate class libraries which while is an effective way to break up the project, is arguably sugar syntax as tight coupling still remains. To be fair though, their agenda was not to create a plug-in system as a primary requirement.

During the past month or so we have been discussing how we could implement a platform that would allow us to easily insert reusable modules of code, or widgets, into various areas of our web systems.  Such a platform should ideally be web technology neutral, so as to allow clients to use ASP.NET xxx, Ruby, plain HTML and with the option of Java.   Ideally it should be a plug-in-based system.

But isn’t that just a CMS?

…I hear you say?  Perhaps.  You could make widgets for a CMS that has arguably the same end result.  But then again, if the system is just a shell and the plug-ins define the look, functionality and behaviour, then a plug-in system could do anything.

A CMS application could be a written using a plug-in-based framework, but not all applications based on plug-in frameworks would be CMSs.  E.g. point-of-sale systems.

So how can we move content from the main web project to a stand-alone DLL that is not directly referenced by the application?  First we need a plug-in system that acts as a mediator between the application and the plug-in.  I wrote my own – the details I cannot disclose here.  Anyway, the problem is not so much the plug-in framework, but how to redirect ASP to your code when looking for a view.  The answer is Microsoft’s .NET class VirtualPathProvider.  This class is quite kewl.  It not only allows you to place MVC content in a different DLL but also anywhere else you want, even in a database!  Apparently many CMSs use this approach.

You need to derive a new class from VirtualPathProvider and override the FileExists and GetFile method’s as documented.  I whacked all this code into my special MVC service (no, not WCF) plug-in.  My framework automatically registers namespace plug-ins (the ones with say Razor stuff) with my service so that when ASP comes along asking for a particular resource, I know which plug-in to grab it from.  I’ve seen a lot of postings saying you must implement your own IViewEngine - that’s not entirely accurate.  It’s only true if you wish to do things like make custom containers in which to place your widgets/views; make a CMS-style app; or smart-clients.  If  you just wish to have say a standard ASP.NET web app albeit with your own plug-ins, then the standard view engines will be just fine.

Now you will find that when creating a class library you will be unable to create say Razor views since it is not a MVC project/application.  The easiest way to fix that is to create them in the web project then move them to the plugin project.  The trick is to mark them as an embedded resource as described here.   Initially I wanted to “pre-compile” them as described here but in my experience I could not get that to work (something about compiling in some ASP namespace), nor were they pre-compiled anyway.  So I just embedded them.

Next I applied what I knew from my fun years writing Windows shell namespace extensions and Microsoft Management Console (MMC) snap-ins.  I really liked these APIs compared to say IoC containers such as StructureMap, mainly due to the way the former uses strongly-typed contract-first patterns compared to the latter’s late-bound reflection behaviour.   I find that when people use StructureMap they are using it like an elaborate class factory anyway with scary configuration.  I find it is better to have zero configuration like the patterns in the Windows shell and MMC.  My framework, which is a service container, automatically enrols and connects plug-in providers to plug-in services.

Not only does our new plug-in framework allow ASP.NET MVC 3 Razor views to reside in external plug-ins, but the main application should it so wish, is able to override the view and still gain access to the plug-ins model!  Yay!  This was an important “must have” that one of my peers expressed to me.

Also, because all the web applications are based on shell namespaces, IShellFolder, IShellView they are presentation agnostic, more so than ASP’s MVC.  We could quickly churn out a native app using the majority of the existing code except for an extra interface for rendering to iOS, WinForms or GNU windows.  The greatest thing is that now our entire website can be reduced to one or more plug-ins with the main app nothing more than a shell.  Think Smart Clients.

I think the new framework is much easier than my first one – ESP.  This current framework is my third generation plug-in system which is the culmination of feedback I have received over the years from my peers.  There is no need for the GAC; the main app only requires a single assembly reference and one line of code to use the framework; and plug-in authors can make use of helper classes and wizards.

I dedicate it to you!

Sadly we ran into a problem which I suspect others have come across at one time or another – it can lead to circular dependencies when one project depends on contracts in another whilst at the same time one component may require say a Windows Communication Foundation (WCF) client proxy.  This is quite significant when many of the components are of the WCF variety.

One solution of course particularly when dealing with Service Orientated Architecture (SOA) is to adopt a Schema First practice whereby you define all your types, messages in XSD format rather than leaping straight into say c#.   Sadly, Schema First can be a bit like designing all your .net classes in a modelling tool where the code is generated – it’s nice but not particularly productive.  Plus, unless you have the rather good Stylus Studio and WSCF.blue tools, who wants to hand-crank XSD and WSDL just to add an extra argument to a method?  Not many agile programmers I don’t think.

Another possibility is to take all your .net domain classes and contracts for all the WCF services and place them in a separate solution.  Though this is arguably faster than code generation from modelling tools or schema first, it is rather annoying when you wish to refactor your code.   Renaming a method in the contracts solution won’t automatically rename the classes in the implementation solutions.

Ultimately having many Visual Studio environments open just for contracts and the services just plain sux, particularly when you need to compile things in a particular order.

So I’ll be boggled if I know what to do for the short to medium term.  I suppose we’ll just continue what seems to be the common practice – have a monolithic Visual Studio solution that at some point in the future we can split up once code changes become idle.

Which opens up another can of worms – splitting up the code introduces a reasonable level of risk particularly when we can assume that its already been QA’d.

I’d love to hear if anyone has some great ways to solve this dilemma.

Making an object cache in .NET is pretty straight forward when you think about it – its just a key-value pair with items that timeout after some time interval.  Making it work over the network is potentially difficult in a decentralised design but WCF 4 has the wonderful NetPeerTcpBinding that does all the tricky work of discovering, joining and leaving a virtual mesh of nodes in the distributed cache via a peer-to-peer (P2P) mesh.  Actually NetPeerTcpBinding was introduced in 3.5 or possibly 3 but WCF 4 is so much easier on the whole but I digress.

Basically NetPeerTcpBinding acts very much like NetTcpBinding so get your program to work with that first.  To transition to NetPeerTcpBinding you merely change your binding – no other changes to program flow are required.

Here’s a snippet of my WCF cache host config file:

<bindings>
         <netNamedPipeBinding>
             <binding name="noSecurityPipeCofig">
                 <security mode="None" />
             </binding>
         </netNamedPipeBinding>
         <netPeerTcpBinding>
             <binding name="noSecurityP2PBinding">
                 <security mode="None" />
             </binding>
         </netPeerTcpBinding>
         <netTcpBinding>
             <binding name="tcpNoSecurityConfig">
                 <security mode="None" />
             </binding>
         </netTcpBinding>
     </bindings>
     <services>
         <service behaviorConfiguration="myServiceBehaviour" name="Schmicky.Cache.Services.CacheService">
             <clear />
             <endpoint address="announcements" binding="netPeerTcpBinding"
                 bindingConfiguration="noSecurityP2PBinding" name="p2pEndpoint"
                 contract="Schmicky.Cache.Contracts.Interfaces.ICacheBroadcast"
                 listenUriMode="Explicit">
           
             </endpoint>
             <endpoint address="local" binding="netNamedPipeBinding" bindingConfiguration="noSecurityPipeCofig"
                 name="pipe" contract="Schmicky.Cache.Contracts.Interfaces.ICacheQuery" />
             <host>
                 <baseAddresses>
                     <add baseAddress="net.p2p://broadcastmesh/SchmickyCache/" />
                     <add baseAddress="net.pipe://SchmickyCache/" />
                 </baseAddresses>
             </host>
         </service>
     </services>

You will note that I have two endpoints.  The reason for this is that the P2P generally only allows for announcements and therefore one-way calls.  This sort of threw a spanner into the works as I thought a cache wouldn’t be too good if you could only store and not retrieve.

Then I thought, well why not make a secondary full-duplex endpoint just for queries?  That would work, and so I made use of the high-speed channel from NetNamedPipeBinding.  It doesn’t matter that its local host only because both endpoints are talking to the local cache access point host.  Yipee!

So with two endpoints it makes sense to map them to two different interfaces – one for storage and the other for cache query and retrieval:

[ServiceContract(ProtectionLevel = ProtectionLevel.None)]
    public interface ICacheBroadcast
    {
        /// <summary>
        /// Puts the specified item.
        /// </summary>
        /// <param name="request">The request.</param>
        [OperationContract(IsOneWay = true)]
        void Put(PutItemRequest request);

        [OperationContract(IsOneWay = true)]
        void Touch(string key);
    }

[ServiceContract( ProtectionLevel = ProtectionLevel.None, SessionMode = SessionMode.Allowed)]
    public interface ICacheQuery
    {
        [OperationContract(IsOneWay = false)]
        GetItemResponse Get(GetItemRequest request);
    }

When an item is stored it is broadcast to all other nodes in the mesh very quickly.  Retrieving an item is immediate for the app storing it in the first place as there is no network latency.  Remote nodes won’t know about it till it is broadcast there.  When any node gets an item I send an asynchronous touch command to all other nodes to keep the item alive.  .NET 4’s Task class is quite useful here.

The best thing about the P2P in WCF is that there is no looping’ing of sending updates to each node – its all encapsulated by WCF. Think UDP.

So now I have a working distributed cache and it only took me a day to do.  Who said things are too complex?

Till next time.