Monday, 10 October 2011

Hosting Dynamic Workflows

I’ve been working with WF for many years now (OK, just over 6 to be precise) and having recently left Microsoft I’m now actually using WF4 in anger.

One of the things I want to do (and judging by the number of posts I’ve found on this I’m not the only one) is the following…

  • Host the Workflow Designer to allow users to create custom workflows
  • Have these custom workflows exposed as services (i.e. .xamlx), hosted using the standard mechanisms inside an ASP.NET site
  • Monitor the workflows using Windows Server AppFabric
  • Do all of the above with the minimum of manual steps (ideally none)

There are a number of hurdles you’ll face when trying to do this, but the point of this post is to show how I have accomplished this with a minimal amount of custom code.

When I approach a problem I’ll often describe an ideal outcome – this helps me to concentrate on the important “must-haves” and also help me know when I’m finished. So, that ideal outcome for this would be…

The user creates a workflow in a custom designer. This workflow is saved in the database and can then be called by a client, using a well-known interface. The address of the workflow will be based on the workflow itself – i.e. the address will also be dynamic.

OK, sounds easy. First off we need to create an editor, I’ve done that in my example application but haven’t described it here as it’s already well documented on the web and in the WCF & Workflow samples. The workflow coming out of the back of the designer is just a piece of text, so if you can’t store that in the database then you’re reading the wrong blog.

So, we have a workflow sat in a database table and need to be able to call it. For sake of argument my database table is as follows…

image

This simply contains a GUID as the key, a textual workflow definition and a textual name. In my example I’ve created a simple EF model that uses this, and exposed that as a very crude but effective WCF service. So, I can get stuff into and out of the table.

There are actually two parts to hosting dynamic workflows, one being having ‘something’ listening on the server, the other is doing the correct thing when a message comes in.

Listening out for a XAMLX

In order to take advantage of the maximum amount of plumbing (i.e. reuse as much code as possible, as an old colleague of mine used to say “less code more smiles”) I wish to host my workflow as a .XAMLX file.

This normally requires that you stuff this file on disk – however, ASP.NET has had the concept of Virtual Path Providers (and by extension, Virtual File Providers) for some time (since ASP.NET 2 if I recall correctly). What a VPP allows me to do is intercept the incoming file request and inject my own code – I can then expose a .XAMLX file without it actually being on disk.

In my example code I’ve created a VirtualPathProvider that is instantiated in global.asax as shown in the code below.

protected void Application_Start(object sender, EventArgs e)
{
// Create and register the virtual path provider
HostingEnvironment.RegisterVirtualPathProvider
(new WorkflowProvider());
}


This then looks for any request that contains the sequence {guid}.xamlx and if found strips off the GUID and looks this up in the Workflow table – the pertinent code is as follows…

private string GetWorkflow(string virtualPath)
{
string definition = string.Empty;

if (virtualPath.EndsWith(".xamlx", true, null))
{
string fileName =
Path.GetFileNameWithoutExtension(virtualPath);
Guid id = Guid.Empty;

if (Guid.TryParse(fileName, out id))
{
using (WorkflowDatabaseEntities context =
new WorkflowDatabaseEntities())
{
var row = context.Workflows.SingleOrDefault
(wf => wf.WorkflowId == id);

if (null != row)
definition = row.WorkflowDefinition;
}
}
}

return definition;
}

OK, so that gets us part of the way but we’re not out of the woods just yet. What we need now is some way to communicate with the workflow. If you were hosting a regular .XAMLX your would define the service contract by dropping activities into the workflow.


That’s all fair and good, however what if you want to be able to communicate with the workflow using a standard, non-changing interface? I should ideally be able to communicate with the Workflow using an interface such as that shown below…

/// <summary>
///
Communicate with a dynamic workflow
/// </summary>
[ServiceContract]
public interface
IDynamicWorkflow
{
/// <summary>
///
Creates a new workflow instance
/// </summary>
/// <param name="args">
Parameters to pass to the workflow
</param>
/// <returns>
The unique ID of the created workflow
</returns>
[OperationContract]
Guid CreateWorkflow(IDictionary<string, object> args);

/// <summary>
///
Resume a bookmark
/// </summary>
/// <param name="workflowInstanceId"></param>
/// <param name="bookmarkName"></param>
/// <param name="state"></param>
[OperationContract(Name = "ResumeBookmark", IsOneWay = true)]
void ResumeBookmark(Guid workflowInstanceId, string bookmarkName, object state);
}

So, this interface allows me to communicate with the workflow – I can create a workflow instance and also resume bookmarks on that instance, much how I’d communicate with something in-process. But how does this interface get associated with the .xamlx workflow definition?


Voodoo Magic


Yes, it’s time to get out the grass skirts, dance around some poor sap tied to a tree and cast some spells. OK, not exactly, but welcome to the world of the WorkflowHostingEndpoint. This class has been created to support just this style of dynamic invocation – all you need to do is derive from it, override a few simple methods and you’re away. Well, nearly.


The most common usage of this endpoint is when you are hosting workflows yourself – i.e. by creating a WorkflowServiceHost instance and specifically adding on the endpoint. This is all documented to a certain extent on MSDN here. But that seems to state that you need to do all this heavy lifting yourself, which is all fair and good but my goal was to minimise the amount of code written – and doesn’t .xamlx activation already do most of this?


The answer is yes, the pipeline that gets invoked when you call a .xamlx will instantiate the WorkflowServiceHost and get all of the plumbing ready, but we need a way to inject our custom endpoint into the mix – and this is where the voodoo magic lies.


To cut a long story short, I’ve spent several hours with Reflector, decompiling the path that a request takes from initially getting to the server in the form of an HTTP request all the way down through the multitude of classes until it reaches some code that actually runs a workflow.


In my quest I was also lucky enough to stumble upon this post which showed me how I could add on a new endpoint within the .config file which was just the job. The configuration elements of interest are shown below…

  <system.serviceModel>

<
extensions
>
<
endpointExtensions
>
<
add name="dynamicWorkflow"
type
="HostingDynamicWorkflows.DynamicHosting.CustomWorkflowEndpointCollection,
HostingDynamicWorkflows
"
/>
</
endpointExtensions
>
</
extensions
>

<
services
>
<
service name="DynamicActivity"
>
<
endpoint kind="dynamicWorkflow" binding="basicHttpBinding" address=""
/>
</
service
>
</
services
>

The first element (an endpoint extension) defines the custom endpoint (well, actually a custom endpoint collection) called ‘dynamicWorkflow’. This is then associated with a dynamic workflow with the second element, the <service> declaration.


At runtime, the underlying workflow hosting code will pickup the <service> element, find the ‘kind’ attribute, and diligently wander off to the class defined in the endpoint extension and instantiate it. This class is the one that derives from WorkflowHostingEndpoint, and also the one that (somewhat indirectly) implements the IDynamicWorkflow interface.


Before I continue however there’s one critical element to all of this which needs to be explained.


When the workflow hosting code starts up your workflow it does the following (in pseudocode)…



string configName = RootActivity.Name;

If (RootActivity Is WorkflowService) && (IsNotNull(RootActivity.ConfigName))
  configName = RootActivity.ConfigName;

// Read configuration information for the workflow from the service element
// with name ‘configName’


With a regular .xamlx workflow you may have noticed there is a property on the root workflow element called ‘ConfigurationName’.


<WorkflowService mc:Ignorable="sap" ConfigurationName="WaitForAWhile" .../>

This maps to the service element…

    <services>
<
service name="WaitForAWhile"
>
<
endpoint kind="dynamicWorkflow" binding="basicHttpBinding" address=""
/>
</
service
>
</
services
>

This tells the workflow hosting code that the configuration for the workflow service is in a named element in the .config file. That’s all fairly easy to understand.


In my config file however this service name is ‘DynamicActivity’ – and this maps to the Name property of the root node of the workflow being executed. You might ask – why is the name “DynamicActivity”. Well, there’s another thing that I’ll describe in the following section.


Passing in dynamic parameters


When starting a workflow I’d obviously like to be able to pass in an arbitrary list of input arguments. In order to do this you need to use the ActivityBuilder class when designing your workflow (this allows you to create arguments, either in code before you display the designer to the user, and/or in the designer itself).


In my client application I have the following code when I’m creating the designer for editing a workflow…


_designer = new WorkflowDesigner();

ActivityBuilder builder = new ActivityBuilder();
_designer.Load(builder);

This then creates a workflow that is persisted as follows (this is one that also contains a message activity)…


<Activity x:Class="{x:Null}"> 
<c:UIMessage Message="Hello from Workflow!" />
</
Activity
>

I have removed all namespaces from the above and other clobber, just to show the bits that are important. If I were to add an argument then the XAML would look something like the following…


<Activity x:Class="{x:Null}">
<
x:Members
>
<
x:Property Name="DateOfBirth" Type
="InArgument(s:DateTime)" />
</
x:Members
>
<
c:UIMessage Message
="Hello from Workflow!" />
</
Activity
>

Now, when this activity is loaded from the database I’m returning it as a file to the workflow hosting mechanism. That is then converting it into a workflow (most probably using ActivityXamlServices.Load(stream). And when this method is used to load a workflow such as the one I’ve defined above, guess what the root node is? It’s a DynamicActivity – and guess what the ‘Name’ property of this class returns – well, that too is ‘DynamicActivity’.


So, in order to load up the appropriate configuration data for my dynamic workflows, I must have a <Service> element within the config file with the name “DynamicActivity”. That then plugs in the endpoint extension, which then allows me to call the workflow using my custom IDynamicWorkflow interface, which then means I’m a happy camper as I can do exactly what I set out to do in the first place.


Example Solution


I’ve created a sample Visual Studio solution and have attached a link at the end of this post. The solution consists of 4 projects as outlined below…



  • A WPF client that allows you to create workflows and then run them.
  • A server project that hosts a WCF service that the client calls to save and edit workflows.
  • A shared project that both client and server use that contains the service contracts and also client proxy classes (I prefer to create these proxies myself when I own both ends of the service, rather than use Add Service Reference).
  • An assembly that contains some custom activities.

In the client application you can create workflows (using “New” on the File menu). Once saved (give the workflow a name in the text box next to the save button) you can then click on a workflow and start an instance. I’ve added two custom activities to the toolbox, a “Message” activity which will display a message on the client UI when the workflow is executed, and a “Wait For Input” activity that creates a bookmark, tells you it’s created it, and then waits for the bookmark to be resumed.


So, this allows you to test the workflow(s) you have created in a rudimentary manner. As an example I created the following workflow that would emit a message at the start, then wait in a loop for one of two bookmarks to be resumed and will complete only if the second branch is executed…


image


Here the workflow outputs a message and the waits for one of two bookmarks to be completed (“Continue” and “Done”). If the Done bookmark is resumed then the loop exits and a completed message is displayed to the user.


When this workflow is executed you’ll see a bunch of log messages in the client application as shown below…


image


This shows I started the workflow, then there are two log messages indicating that a bookmark was created, then a message indicating that Continue ‘completed’ (meaning I resumed the ‘Continue’ bookmark from the UI). This loops back again to re-create the bookmarks, and I finally resumed the ‘Done’ bookmark which completed the workflow.


If you’re wondering, the client application also exposes a WCF service that the Web site hosting the server portion uses to notify these changes. That’s just for this example, it’s probably not a good idea for a real server application!


The Code


Please click here to download the code. If it works it was written by me, if it doesn’t I don’t know who wrote it. Smile

8 comments:

obaid said...

This is a very good article. I have scenario to implement very much similar to what u have described in the article but with little variance. My Scenario is as follows,

1. Create a workflow in custom designer.
2. Store it in database.
3. Expose a WCF service via a windows service.
a. This service exposes a single call as RunWorkflow(IEntity obj).
4. Based on the information provided in IEntity, system should be able to pull the workflow from the database and execute (this would be a long running workflow). This workflow execution should be managed by somebody else.
a. There could be 100's of calls to Runworkflow method - that means there could hundred's of long running workflows running simultaneously.
5. Monitoring/Tracking/Persistance should also be provided.

1-3 is trivial. My confusion is how do I get 4-5 working. Please guide me in this direction.

obaid said...

This is a very good article. I have scenario to implement very much similar to what u have described in the article but with little variance. My Scenario is as follows,

1. Create a workflow in custom designer.
2. Store it in database.
3. Expose a WCF service via a windows service.
a. This service exposes a single call as RunWorkflow(IEntity obj).
4. Based on the information provided in IEntity, system should be able to pull the workflow from the database and execute (this would be a long running workflow). This workflow execution should be managed by somebody else.
a. There could be 100's of calls to Runworkflow method - that means there could hundred's of long running workflows running simultaneously.
5. Monitoring/Tracking/Persistance should also be provided.

1-3 is trivial. My confusion is how do I get 4-5 working. Please guide me in this direction.

Morgan said...

obaid,

Sorry for my tardiness getting back to you, I wasn't notified that there was a comment here and have been soo busy working that I've not checked.

My example does exactly what you need - albeit not using an IEntity interface. It allows you to start any number of workflows and call back into these at will.

Hope that makes sense,

Cheers,

Morgan

nitinverma said...

Hi, This is a fantantic article which fits 90% of my requirement. The one thing can't understand is how do we get the output/result from an activity? Should we use WCF service to get the value..If yes then where we can put the code to get it as we don't have control over WorkflowHost.

Nitin

musicman said...

...Awesome post, and thank you for taking the time to publish all these articles. I actually was in the middle of trying to design a 'dynamic' workflow solution when I found this particular post. I really like the idea of leveraging 'standard' XAML-only workflows without having users configure WCF endpoints, Receive activities or use custom container activities just to run in IIS. The techniques outlined here are really powerful and might allow for a more dynamic workflow solution (with App Fabric support) for 'non-engineering' users.

Question: On security- How do I access the "OperationContext" from my activities? I have leveraged your techniques and I would like to leverage WCF security, and couldn't seem to find the solution. The ultimate goal is to be able to set the Thread.CurrentPrincipal so I can access the user info in tracking and persistence participants to tell 'who did what' from the application or directly from App Fabric. In my hosting endpoint, I have been able to leverage WCF security behavior (using IAuthorizationPolicy and ServiceAuthenticationManager). I am able to add 'custom principal' information to the 'System.ServiceModel.OperationContext', but unfortunately OperationContext gets lost in the asynchronous WF execution. According to the MSDN documentation (http://msdn.microsoft.com/en-us/magazine/gg598919.aspx), the standard way to access OperationContext using WF services is to implement a 'scope' style activity adding an execution property with a IReceiveMessagecallback. The problem is, I am *not* explicitly using a 'Receive' activity with the WorkflowHostingEndpoint, and therefore, the IReceiveMessagecallback does not fire.

Is there a way to access OperationContext without an explicit "Receive" activity?

Thanks in Advance.


[Designer(typeof(SecurityScopeDesigner))]
public sealed class SecurityScopeActivity: NativeActivity
{
public Activity Body { get; set; }

protected override void Execute(NativeActivityContext context)
{
if (Body == null) return;

context.Properties.Add(ContextInspector.Name, new ContextInspector());
context.ScheduleActivity(Body,new CompletionCallback(MyActivityCompletionCallback));
}

private void MyActivityCompletionCallback(NativeActivityContext context, ActivityInstance completedInstance)
{
//complete logic
}

}

[DataContract]
internal class ContextInspector : IReceiveMessageCallback, IExecutionProperty
{
private OperationContext _current;
private OperationContext _original;
[DataMember]
public IPrincipal Principal { get; set; }
public static readonly string Name = typeof(ContextInspector).FullName;


public void OnReceiveMessage(OperationContext operationContext, ExecutionProperties activityExecutionProperties)
{
//get the operation context and set the operationContext.current ...
//_current= ... get from operation context
}

public void SetupWorkflowThread()
{
OperationContext.Current = _original;
//todo: parse operation context...
//todo: set the System.Threading.Thread.CurrentPrincipal...
}

public void CleanupWorkflowThread()
{
_original = OperationContext.Current;
OperationContext.Current = _current;

}
}

Marius Mössmer said...

Awesome!
this post describes indeed a beautiful solution for my problem.

but the sourcecode-package (http://www.morganskinner.com/Articles/DynamicWorkflows/HostingDynamicWorkflows.zip) isn't online anymore. Could you publish it once again?
greets, Marius

Dan said...

Looks like he's popped it onto:
http://msadcworkflow.codeplex.com

No source code though! BAH!

Blogger said...

Bluehost is one of the best hosting company with plans for all of your hosting requirments.