Failover and Its Customization

Default Recovery

In the case of a server unscheduled shutdown, in the database, there can appear processes stuck in the Running status. This is the only process status that blocks any other operations with the process. The recovery procedure finds such processes and resets their statuses to * Error* or Terminated.

The Error status means that the process can be changed with command triggered transitions or timer triggered transitions. Thus, the process does not hang in this state, and you can work with it as before.

The Terminated status means that the process has no exit either by command or by timer. Thus, you can restore the execution of this process, for example, using the workflowRuntime.SetState() method. That is, such a process requires attention.

The recovery does not differ, whether in the single-server mode or the multi-server mode. In the single-server mode, each process with the Running status is restored after the server is restarted; in the multi-server mode, only those processes are restored, whose Running status was set by the runtime selected for the recovery.

The Error and Terminated statuses are set automatically and do not require configuring.

Recovery of Timers

Timers are not subjects to recovery, because they will always be processed if at least one server is running.

Customization of Recovery Procedure

To customize the recovery procedure after a shutdown, you should write Process Restorer, implementing the IProcessRestorer interface:

public interface IProcessRestorer
{
    bool NeedCustomRestore(string schemeCode, ProcessStatus status);
    RestoreDecision GetCustomRestoreDecision(ProcessInstance process);
}

Let us consider its methods.

• NeedCustomRestore - should return true if the custom recovery procedure is necessary. status passed to it is either Error or * Terminated*, automatically set for the process on recovery. schemeCode is the process scheme code. If the method returns true, then the GetCustomRestoreDecision method will be called for each process to be restored.

• GetCustomRestoreDecision - process Instance is passed to the method, containing all the information about the process; in the result, the method returns the decision of what to do next with this process. The following decisions are possible:

  • return RestoreDecision.Delete - delete the process.
  • return RestoreDecision.RunFromCurrentActivity - restart the process with the previous Activity. Please, note that in the case of a failure, the running Activity has not been fixed yet, so the process will start with the previous Activity. That is all right. Read about the process execution here.
  • return RestoreDecision.RunFromNewActivity(activityDefinition) - start the process with a new Activity, activityDefinition is an element of the process scheme.
  • return RestoreDecision.Default - apply the default recovery mechanism, that is, set either Error or Terminated status.

Moreover, using RestoreDecision, the process parameters can be changed, which is a powerful tool to manage the process execution.

For example, the Process Restorer can look as follows:

private class YourCustomProcessRestorer : IProcessRestorer
{
    public bool NeedCustomRestore(string schemeCode, ProcessStatus status)
    {
        return schemeCode == "SomeSchemeCode" && status == ProcessStatus.Terminated;
    }
  
    public RestoreDecision GetCustomRestoreDecision(ProcessInstance process)
    {
        return RestoreDecision
            .RunFromNewActivity(process.ProcessScheme.InitialActivity) // set to Initial Activity
            .AddPersistentParameter("wasFailed", true); // the sign that the process was failed
    }
}

After creating a customized Process Restorer, it should be connected to Workflow Runtime.

var runtime = new WorkflowRuntime()
    ...
    .WithProcessRestorer(new YourCustomProcessRestorer())
    .AsSingleServer(); // .AsMultiServer();

Any number of Process Restorer instances can be connected to one runtime. The one, whose NeedCustomRestore method returns true for the process selected for recovery, will be used.

info

Please, note that Process Restorer is only used in the case of a server shutdown. The OnWorkflowError event handler might be used for the workflow errors.