Temporal Workflow

A Workflow defines a sequence of steps. With Temporal, those steps are defined by writing code, known as a Temporal Workflow Definition, and are carried out by running that code, which results in a Workflow Definition.

A Workflow is the coordination of your system.

There's no retry system on the Workflow level.

1 Workflow Definition combined with n Workflow Requests results in n Workflow Execution.

Workflow Definition

A Workflow Definition is the code that defines the constraints of a Workflow Execution.

Transclude of 2024-07-22-temporal-workflow_composition.excalidraw

Implementation in java

Add Temporal annotations to the class/interface you want it to be a Workflow Definition:

package io.temporal.learn;
 
import io.temporal.workflow.WorkflowInterface;
import io.temporal.workflow.WorkflowMethod;
 
@WorkflowInterface
public interface Greeting {
 
    @WorkflowMethod
    String greetSomeone(String name);
}

Workflow Execution

A Temporal Workflow Execution is a durable, scalable, reliable, and reactive function execution. It is the main unit of execution of a Temporal Application.

Workflow Execution States

+------+     +--------+
| Open | --> | Closed |
+------+     +--------+

This is one-way transition. Every Workflow Execution has a unique Run ID.

There is only ONE Open State, but there are several Closed States:

• Completed
• Continued-As-New
• Failed
• Canceled
• Terminated
• Timed Out

Transclude of 2024-07-23-temporal-workflow_execution_states.excalidraw

Sticky Execution

Workers cache the state of the Workflow they execute. To make this caching more effective, Temporal employs a performance optimization known as “Sticky Execution,” which directs Workflow Tasks to the same Worker that accepted them earlier in the same Workflow Execution.

Terminating a Workflow Execution

From command line:

temporal workflow terminate \
    --workflow-id process-invoice-82461092
    --reason "Invalid input data provided when starting the Workflow"

Programmatically:

import io.temporal.client.WorkflowClient;
import io.temporal.client.WorkflowOptions;
import io.temporal.serviceclient.WorkflowServiceStubs;
import io.temporal.client.WorkflowStub;
 
public class Terminator {
 
  public static void main(String[] args) throws Exception {
    String workflowId = "pizza-workflow-order-XD001";
 
    WorkflowServiceStubs service = WorkflowServiceStubs.newLocalServiceStubs();
    WorkflowClient client = WorkflowClient.newInstance(service);
 
    PizzaWorkflow workflow = client.newWorkflowStub(PizzaWorkflow.class, workflowId);
    WorkflowStub stub = WorkflowStub.fromTyped(workflow);
 
    stub.terminate("Invalid input data provided when starting the Workflow");
    System.exit(0);
  }
}

Workflow ID

When starting a Workflow, you specify a Workflow ID that is associated with that execution. A Workflow ID is intended to be meaningful for your business logic. For example, an order processing Workflow might use a Workflow ID that includes the order number (e.g., process-order-number-90743812), while a Workflow that manages the loan that a bank has made to one of their customers might include the account number (e.g., manage-loan-account-28430614).

// Example: An order processing Workflow might include order number in the Workflow ID
WorkflowOptions options = WorkflowOptions.newBuilder()
        .setWorkflowId("process-order-number-" + input.OrderNumber)
        .setTaskQueue(Constants.taskQueueName)
        .build();
 
ProcessOrderWorkflow workflow = client.newWorkflowStub(ProcessOrderWorkflow.class, options);

Temporal guarantees that there can only be a single Workflow Execution with a given Workflow ID running within a Namespace at any point in time (a Namespace is a logical unit of isolation, often used to separate Workflows running on the same cluster but owned by different teams).

This constraint applies to Workflow Executions of any type within that Namespace, not just those of the same type. For example, an execution of the order processing Workflow cannot use the same Workflow ID currently used by a running execution of the loan management Workflow. This is an important consideration when choosing a Workflow ID value.

How Temporal handles a Workflow ID conflict

In the Java SDK, an attempt to start a Workflow Execution while one with the same Workflow ID is already running is effectively ignored. Rather than start a new Workflow Execution, it will return the information associated with the one that is already running, although you can configure WorkflowOptions to instead return an error in this case.

Workflow ID reuse policy

Temporal guarantees that there can only be a single Workflow Execution with a given Workflow ID running within a Namespace at any point in time. Temporal allows you to further constrain the uniqueness of the Workflow ID within the Namespace by specifying a Workflow ID Reuse Policy when you start a Workflow. This policy allows for one of four options that govern whether the Workflow ID can be reused:

• WORKFLOW_ID_REUSE_POLICY_ALLOW_DUPLICATE
  • This is the default, which allows the Workflow ID to be reused by another Workflow Execution in the same Namespace upon the conclusion of the current one, regardless of whether that resulted in success or failure.
• WORKFLOW_ID_REUSE_POLICY_ALLOW_DUPLICATE_FAILED_ONLY
  • This policy is similar to the default, but only allows the Workflow ID to be reused if the Workflow Execution that used it did not complete successfully.
• WORKFLOW_ID_REUSE_POLICY_REJECT_DUPLICATE
  • This policy does not allow the Workflow ID to be reused at all, regardless of how the previous one ended.
• WORKFLOW_ID_REUSE_POLICY_TERMINATE_IF_RUNNING
  • This policy is similar to the default, with one important difference: It terminates the Workflow Execution currently using the specified Workflow ID, if there is one. This ensures that the new Workflow Execution will be the only one running in the Namespace with the specified Workflow ID.

Retention period

One thing to keep in mind if you specify a more restrictive Workflow ID Reuse Policy is that Temporal can only enforce uniqueness based on the Workflow Executions for which it still has data. In order to maintain good performance and conserve storage space used by the cluster’s database, data associated with past Workflow Executions is eventually deleted. This duration between when a Workflow Execution has finished and when it is eligible for automatic deletion is known as the Retention Period. It typically has a value between 1 and 30 days. Although it may be possible to specify a longer retention period if you’re running a recent version of the Temporal Cluster software, this is generally discouraged because it can increase storage requirements and may diminish performance.

Some users are subject to compliance regulations or other requirements that mandate the ability to determine what took place during a Workflow Execution several months or even years after it concluded. Increasing the retention period may seem like an attractive solution for these use cases, but Temporal provides an alternative solution.

Self-hosted Temporal clusters provide the Archival feature. When Archival is enabled, data associated with a Workflow Execution is automatically copied to an appropriate long-term storage system (such as a local filesystem or Amazon S3) before being deleted after the Retention Period is reached.

Temporal Cloud provides a similar feature, known as Export. After Export is initially configured, it will export history data from your namespace to S3 on an hourly basis.

Retention period do not affect open Workflow

Be sure to note that the countdown for the Retention Period only begins once the Workflow Execution finishes. The Retention Period does not affect Workflow Executions that are currently running, so a five-day retention period won’t cause any problems for a Workflow Execution that runs for 10 years. Data associated with a running Workflow Execution is always available, regardless of how long that execution runs.

Workflow Task

Transclude of 2024-07-23-temporal-workflow_task_states.excalidraw

Considerations

• Inputs and outputs MUST be serializable, so they can be stored in Temporal as part of the Event History
• Data confidentiality:
  • You can create a custom Data Converter to encrypt the data as it enters the Temporal Cluster and decrypt it upon exit.
  • Reference: Temporal Platform security | Temporal Documentation
• Limit the amount of data sent, e.g. instead of passing the whole video file, you can pass the URL to access to the video instead.

Determinism / Idempotence

Workflow code must be deterministic!

During History Replay, the Worker uses Events to recover the state of the previous execution.

This deterministic aspect gives Temporal the ability to guarantee durable execution of your Workflow.

Common sources of non-determinism:

• using random numbers
• accessing / mutating external systems or state, e.g. databases, file systems, network services, …
• relying on system time
• working directly with Thread
• iterating over data structures with unknown ordering
• storing or evaluating the Temporal Run ID
  • The Temporal Run ID is mutable and can change if the Workflow is retried.

Examples of changes that may lead to non-deterministic errors:

• adding or removing an Activity,
• switching the Activity Type used in a call to an Activity Method,
• adding or removing a Timer,
• altering the execution order of Activities or Timers relative to one another.

Examples of changes that do not lead to non-deterministic errors:

• modifying statements in a Workflow Definition, such as logging statements, that do not affect the Commands generated during Workflow Execution,
• changing attributes in a ActivityOptions or RetryPolicy,
• modifying code inside of an Activity Definition.

More on Idempotency and Durable Execution | Temporal Technologies.

Using Workflow Reset to recover from a bad deployment

One way is to use the workflow reset to recover:

$ temporal workflow reset \
        --workflow-id pizza-workflow-order-XD001 \
        --event-id 4 \
        --reason "Deployed an incompatible change (deleted Activity)"

Other options are:

• cancel the Workflow,
• terminate the Workflow.

Then re-trigger the Workflow using the same inputs.