In late 2022, we were approached by a large customer in the automotive industry who asked us to help them implement a\nself-service solution with the ability for bi-directional communication to their PLCs (Programmable Logic Controllers).\nBefore our engagement with them, PLC operators would need to manually configure each of the relevant PLCs on the factory\nfloor to obtain desired output\/results. Our project helped eliminate or simplify these manual steps, which in turn\nhelped the customer iterate and scale production faster, while potentially cutting down on manual errors.<\/p>\n
Given that production of such hardware components requires a series of steps to be executed in sequence as a workflow<\/em>,\nwe found that workflow engines would be good candidates to base our solution upon. From the variety of existing workflow\nengines we selected the Durable Task Framework (DTFx)<\/a> due to its performance\ncharacteristics, broad capabilities, big community and Microsoft support.<\/p>\n However, out-of-the-box DTFx also did not meet all of our requirements and we built some features on top of it. With this\nblog post, we detail what we built, explaining the why and how. We hope this information will be helpful if you are\nconsidering using DTFx and want to tailor it to your specific needs.<\/p>\n A fundamental requirement of the solution was to be self-service<\/em> in the sense that factory operators should be able\nto define workflows to cover future use-cases. Given that operators often lack coding skills, we addressed this\nrequirement with the introduction of a Domain-specific language (DSL)<\/a> that acts as an\nabstraction layer, enabling operators to create workflows in an easy and user-friendly way.<\/p>\n Another requirement was to be able to influence the workflow execution based on input provided externally at workflow\nexecution time or depending on values generated while running the workflow e.g., the current value of a PLC node. To\naddress this we introduced dynamic expressions and a data-flow<\/a> to pass data from a\nworkflow step to subsequent steps.<\/p>\n The workflows we are dealing with have (write) access to machines on the factory floor, so validation of dynamic\nexpressions<\/a> and the workflow as a whole<\/a> is crucial to\nensure safety and communicate issues earlier to factory operators.<\/p>\n In some cases, workflows could take a long time to be completed or even all together hang. This could happen due to\nvarious reasons, like an incorrect information in a workflow configuration or transient network issues on the factory\nfloor. To avoid this and recover gracefully, we provided a way to handle workflow timeouts and\ncancellations<\/a>.<\/p>\n In other cases, workflows need to execute a certain “cleanup” action independently of the result of the execution. To\ncover this requirement we added a workflow closure step<\/a>.<\/p>\n In DTFx workflows are exclusively defined through code. To empower factory operators with the ability to define workflow\nsteps without coding, we recognized the need for a Domain-specific Language (DSL). Besides the fact that operators often\nlack coding skills, adding workflows via code would also be error-prone, so we aimed to create a DSL that would act as\nan abstraction layer, enabling operators to create workflows in an easy and user-friendly way. As an example, operators\nwouldn’t have to enter detailed information on how and where PLC nodes can be reached because this information was\n“enriched” from our backend, minimizing the workflow definition inputs required of operators.<\/p>\n Although, eventually end users will use a UI to interact with the solution, which will generate the underlying workflow\ndefinition JSON, having a well-designed DSL was important to onboard users fast and even before the UI was ready. It\nwould also support more advanced future use-cases like workflow definition versioning and having definition JSON\ngenerated by external systems.<\/p>\n To achieve this, the concept of workflow definition<\/em> was separated from workflow configuration<\/em>. You can think of the\ndefinition being a “compile-time” construct which uses user-facing terms like signals, versus configuration being a\n“runtime” construct, containing all details needed to execute the workflow.<\/p>\n As for DSL base language, we chose JSON over Yaml due to easier writing and better support from C# libraries. The fact\nthat DTFx uses JSON internally to serialize its state, was another reason to choose JSON.<\/p>\n As we saw previously, the workflow definition time is not the same as the workflow execution time. We can think of it as\ncompile-time vs runtime. With the term dynamic expressions, we refer to expressions that are evaluated at “runtime”,\ni.e., when a workflow is executed. This allows the PLC operators to influence the workflow execution based on input\nprovided externally at execution time or values generated while running the workflow e.g., the current value of a PLC\nnode.<\/p>\n \u2755 Tip: Verify whether a static workflow configuration is sufficient for your business needs or whether workflow\nexecution could vary depending on runtime parameters. Need for runtime arguments and dynamic expressions might\ninfluence the overall design of the engine, so it’s highly recommended to identify these needs as early as possible.<\/p><\/blockquote>\n To support dynamic execution of workflows, we relied heavily on Dynamic Linq<\/a> functionality\nand more specifically for:<\/p>\n One example of dynamic expressions are control structures which are an essential part of any programming language,\nincluding a custom DSL. Control structures allow end-users (in our case PLC operators) to specify conditions, loops, and\nbranching statements, among others, that dictate how a workflow would execute. The exact type and number of control\nstructures for your custom DSL will depend on its intended purpose and the business needs.<\/p>\n \u2755 Tip: Consider which control structures make most sense for your use-case and are both expressive and easy to use for\nyour end-users and start with those.<\/p><\/blockquote>\n In our use-case we identified if-conditions as a fundamental control structure to start our implementation from. We\nrelied on Dynamic Linq DynamicExpressionParser.ParseLambda<\/a> method that\ncreates a\nLambdaExpression<\/a> out of\na condition provided as a string. This LambdaExpression can then be compiled to create a The execution plan of a workflow could be influenced by input parameters on execution time or by values that were\ngenerated from previous steps of the workflow. Both of these were stored as properties in an instance of\n Assuming the workflow inputs are represented using the following class:<\/p>\n The following sample code shows how we populated the workflowData for the input parameters. A similar approach was used\nto pass outputs of previous steps to the next steps of the workflow.<\/p>\n Validation of dynamic expressions is crucial in order to provide early feedback to the user in case of errors or prevent\nexecution of unwanted code snippets which could constitute a security issue.<\/p>\n \u2755 Tip: Consider validation if you are allowing dynamic expressions in your workflows to ensure no malicious code can\nbe executed and errors can be communicated to the end-user early.<\/p><\/blockquote>\n In our case, syntactic validation (e.g. a step is attempting to use data from a previous step that does not exist) is\nhappening via Dynamic Linq DynamicExpressionParser.ParseLambda<\/a> that\nthrows a Regarding validation from a security point of view, Dynamic Linq already restricts the attack surface by allowing access\nonly to a pre-defined set of types<\/a>: only primitive\ntypes and types from the Further restrictions can be introduced by using the Visitor pattern<\/a> and\nextending the\nDynamicExpressionVisitor<\/a>. For\nexample, in order to check method calls the following snippet can be helpful. The same pattern can be used to check\noperators, fields, properties etc.<\/p>\n Besides the validation of expressions that we just covered, validation of the whole workflow definition helps build a\nmore user-friendly engine by detecting possible errors or misconfigurations and providing feedback to end-users early.<\/p>\n \u2755 Tip: Consider validating as early as possible to give feedback to end-users. In this feedback you can potentially\ninclude all issues found, to allow for resolution of them at once.<\/p><\/blockquote>\n Some basic validation is already provided if you rely on concrete types instead of a generic What is more, instead of primitive types (like In some cases, workflows could take a long time to be completed. This could happen due to various reasons,\nlike an incorrect information in a workflow configuration or transient network issues on the factory floor.<\/p>\n In our specific customer scenario, where the workflow engine runs on a factory edge, workflows need to finish\nand free up system resources as fast as possible. Having a workflow running for a long time without any response\nis undesired and would in most cases mean some kind of misconfiguration or network issue.\nTo handle this case, we were asked to give a user an opportunity to specify a timeout value<\/em> for the entire workflow.<\/p>\n To implement this feature we followed the approach recommended in the\nofficial documentation for Azure Durable Functions<\/a>.<\/p>\nBrief overview of requirements and mapping to the features we built<\/h2>\n
Contents<\/h2>\n
\n
Domain-specific language (DSL)<\/h2>\n
Dynamic expressions and data flow<\/h2>\n
\n
Dynamic expression example: if-condition<\/h3>\n
Delegate<\/code> that can be invoked\nusing the DynamicInvoke<\/a> method on\nit, as shown below.<\/p>\nvar condition = \"1 == 1\";\r\nDelegate parsedDelegate = null;\r\ntry\r\n{\r\n LambdaExpression parsedCondition = DynamicExpressionParser.ParseLambda(\r\n new ParsingConfig(),\r\n Array.Empty<ParameterExpression>(),\r\n typeof(bool),\r\n condition);\r\n parsedDelegate = parsedCondition.Compile();\r\n}\r\ncatch (Exception e) when (e is ParseException or InvalidOperationException)\r\n{\r\n \/\/ handle any exceptions during parsing the condition\r\n}\r\n\r\nif ((bool)parsedDelegate?.DynamicInvoke(null))\r\n{\r\n \/\/ execute then branch\r\n}\r\nelse\r\n{\r\n \/\/ execute else branch\r\n}<\/code><\/pre>\nData flow: workflow inputs and runtime data<\/h3>\n
System.Linq.Dynamic.Core.DynamicClass<\/code>.<\/p>\nclass WorkflowInputParameter\r\n{\r\n public string Name;\r\n public string Type;\r\n}<\/code><\/pre>\n\/\/ parameters as defined in compile-time\r\nvar workflowParameters = new List<WorkflowInputParameter>()\r\n{\r\n new () { Name = \"temperature-threshold\", Type = \"System.Int32\" }\r\n};\r\n\r\n\/\/ convert to DynamicProperties\r\nList<DynamicProperty> dynamicProperties = workflowParameters.Select(p => new DynamicProperty(p.Name, Type.GetType(p.Type))).ToList();\r\n\r\n\/\/ create DynamicClass to hold values\r\nType workflowType = DynamicClassFactory.CreateType(dynamicProperties);\r\nDynamicClass workflowData = (DynamicClass)System.Activator.CreateInstance(workflowType) !; \/\/ consider handling nulls explicitly\r\n\r\n\/\/ actual values passed in runtime\r\nvar parameterValues = new Dictionary<string, int>()\r\n{\r\n {\"temperature-threshold\", 14}\r\n};\r\nworkflowParameters.ForEach(\r\n p => workflowData.SetDynamicPropertyValue(p.Name, Convert.ChangeType(parameterValues.GetValueOrDefault(p.Name), Type.GetType(p.Type))));<\/code><\/pre>\nValidation of dynamic expressions<\/h3>\n
ParseException<\/code> as shown in the previous snippets. This can happen among others when a member does not exist,\nor a parameter is not of proper type.<\/p>\nSystem.Math<\/code> and System.Convert<\/code> namespaces are accessible. This can be configured\/extended\nusing the DynamicLinqType attribute<\/a> on a custom type.<\/p>\n\/* code omitted for brevity *\/\r\n\r\n\/\/ expression as a string\r\nstring expression = \"Convert.ToInt32(data.ReadOpc1.Signals[\\\"signalB\\\"].Value) > data.IntParam\";\r\nLambdaExpression parsedExpression = DynamicExpressionParser.ParseLambda(new[] { dataParam }, typeof(object), expression);\r\n\r\n\/\/ initiate expression visit\r\nWorkflowExpressionVisitor visitor = new ();\r\nvisitor.Visit(parsedExpression);\r\n\r\n\/\/ Visitor class\r\npublic class WorkflowExpressionVisitor : DynamicExpressionVisitor\r\n{\r\n protected override Expression VisitMethodCall(MethodCallExpression node)\r\n {\r\n \/\/ perform any required checks e.g. that the declaring type\r\n \/\/ or the method called is whitelisted\r\n\r\n return base.VisitMethodCall(node);\r\n }\r\n\r\n}<\/code><\/pre>\nWorkflow validation<\/h2>\n
object<\/code> when you\ndeserialize the workflow definition. We relied also on attributes to specify required JSON properties and implemented\ncustom validation based on business rules (using FluentValidation in C#).<\/p>\nint<\/code> or string<\/code>) consider using custom types for fields that have\nbusiness value or hold domain semantics. This way if there is a need to refactor these in the future due to business changes,\nyou can better encapsulate changes.<\/p>\nWorkflow timeout and cancellation<\/h2>\n