In this video, we are going to learn how a list functions orchestrate distributed transactions with
using Python.
I will give some examples an implementation with self function and showing these architecture diagram
on the slide.
So basically you can think about that when we place an order to order, fulfillment process is starting.
So think that we have to microservices, order and fulfillment microservices.
When we place an order order microservices.
Note five fulfillment microservices to start fulfillment processes.
We call this communication is an event notification pattern, but order fulfillment process has several
steps.
For example, if the payment service is still pending, when the order arrives to fulfillment, that
will be a problem.
Or it could be a short delay in the payment status update in the database.
This problem can be solved with the wait state in a list step functions.
Even notification is file and forget pattern.
It emits an event and execute the process but doesn't allow the users to control the timing.
So how we can manage delays by using event driven microservice architectures.
This can very easily solved with a list of functions.
Edible functions has weight state.
When the source services emits an event, the workflow waits for 10 seconds in this case and then starts
the fulfillment processing steps.
As you can see that with step functions, we can get more control over the service integrations by creating
state machines and workflows on a list of functions.
So microservices usually communicate through events with asynchronous communications to indicate the
process change.
So in our ecommerce case, customer placed an order in our eCommerce application API gateway next to
request to order microservices.
We can see in here and order microservices get processed and forward to inventory service and then payment
service and provide to fulfillment process.
This is the happy path of place order use case in ecommerce domain.
There are no failures on the workflow and order get placed successfully.
You got to place order request and you can continue to follow these microservices and perform the fulfillment
process and success to order.
So this is an example of distributed transaction with good persistence to all the transaction.
Data gets stored across different databases and each service writes its own databases.
What if we faced a network failure and the payment gateway is timeout?
We can see in here what if the payment gateway is done and has timeout?
So the last step of this place order workflow has failed at the time of failure.
The order database and the inventory database are all in the updated to order status is set to fail
in order to indicate failure to the downstream services.
But if you look at the our order data, it is now inconsistent and need to roll back updates at that
pace.
Order status is set as ordered and inventory has been documented, so there is no way to correct data
and there is no option to retry the process that have failed.
So how we can fix this problem in our workflow?
If you design monolithic application with relational databases, we can provide transactional automaticity
consistency with for in case if a transaction pays in a relational database the transaction rollback.
But when it comes to distributed transaction in microservice architectures, we can't use database commits
and transactions due to transaction data is distributed across various databases.
In this case, the solution is use the circle pattern with using circle pattern.
If any transaction pace in the workflow, the circle executes a series of compensating steps that roll
back the chains that were made by the preceding transactions.
So if you think about our main use case, when we place order case in the ecommerce application, customer
placed an order and when the item was deducting in stock, another customer order gets fulfilled.
In the meantime, it goes out of stock and after the order has been placed.
So as we all know that this can happen when multiple customers are trying to purchase the same item
at the same times.
So in this case, the update inventory step will fail in the workflow.
The orchestrator will now execute the compensatory steps.
It will run rollback inventory.
And remove the order steps and return status.
The A's failed.
These compensatory steps are provided to ensure the data, integrity and consistency is maintained.
The event rewards the original level and the orders reverted back.
As you can see that the every action that makes a change to the databases there is an opposite action
to compensate the change in case of a failure.
So a failure that happens in the last step, the payment service in this case, we saw that in this
case, the payment is done.
This will trigger all the compensatory transactions to be executed before returning to failure state.
We can implement this algorithm flow with using a step function workflows.
The individual steps invoke lambda function to perform the task.
The step transitions are defined in the Amazon State language, so it is very easy to set up and configure
the compensatory transactions in case of failures.
We will use such orchestration to implement e-commerce place or the use case for our happy and failure
scenarios.
So we will create an API gateway to trigger or add step function when place order request comes from
customer and saga orchestrator pattern implementation with the ADA step functions in order fulfillment
orchestration, we will use success and failure pets in a distributed transactions, restoring data
consistency in Amazon dynamic databases.
So now let's get started.