Async Service in TrueFoundry allows us to put input requests in a queue, process them asynchronously and then put the outputs in another queue. You can use an async service if one or more of the following scenarios applies to your use case:
To build an async service, we need to provision an input queue where we will be writing the messages. TrueFoundry supports the following queues for input queues. Each of the queue points to a page on how you can provision the queue.
You can consume messages from the queue either via our async library or via a sidecar.
Since an async service consumes messages from a queue, we will need the logic to pull and push items to the queue. While you can write this logic in your own code, it can be cumbersome and you will have to change your code if you ever decide to switch the queue system underneath. To overcome this, TrueFoundry provides a simple open-source Python library that integrates with different queue frameworks and you just need to implement the processing handler. You can follow the README in the Github repository to integrate it into your codebase.
For all other use cases, the approach provided below using a sidecar is recommended.
In this case, the python library mentioned above is packaged into another docker image and runs alongside your main code in a separate container as a sidecar. You will need to write your processing code as HTTP service.
The tfy-async-sidecar component consumes items from the queue and calls your HTTP service with the payload as a POST request. The tfy-async sidecar code is open source and available here. It has adapters to consume the messages from the different types of queue as mentioned above. If you need support for additional queues please feel free to reach out, or contribute the queue adapter on this repository.
Once the sidecar pulls the message from the queue, it needs to deliver it to the HTTP service which is written by the user. There are no constraints on the HTTP service and you can write it using your own preferred framework like FastAPI, Flask, ExpressJS, etc. The service needs to expose an endpoint where it will accept the messages in the queue as the body in the POST request. You can provide that endpoint as an input in the async service deployment spec.
After the HTTP service receives the message, it processes it and returns the response to the sidecar. If we configure an output queue, the sidecar can then go and write the data back to the output queue. You can also handle the writing in your service code and not add the output queue configuration if you want.
The sidecar acks the message to the input queue once it has received the response from the HTTP service and written the response to the output queue. If there is a failure at any of the intermediate steps, the queue will redeliver the message to one of the replicas again for processing. This ensures that there is a high level of reliability for the input messages to be processed.
If you’re new to the TrueFoundry Async Service, follow the comprehensive guide given below on how to deploy your service as an asynchronous service. Please make sure you have deployed a simple service on TrueFoundry before embarking on this tutorial. You will need to have the following components ready before deploying an async service.
You can follow Deploy your first Async Service guide to understand the basics of deploying Async Service.
You can configure the settings further using the guides below:
Async Service in TrueFoundry allows us to put input requests in a queue, process them asynchronously and then put the outputs in another queue. You can use an async service if one or more of the following scenarios applies to your use case:
To build an async service, we need to provision an input queue where we will be writing the messages. TrueFoundry supports the following queues for input queues. Each of the queue points to a page on how you can provision the queue.
You can consume messages from the queue either via our async library or via a sidecar.
Since an async service consumes messages from a queue, we will need the logic to pull and push items to the queue. While you can write this logic in your own code, it can be cumbersome and you will have to change your code if you ever decide to switch the queue system underneath. To overcome this, TrueFoundry provides a simple open-source Python library that integrates with different queue frameworks and you just need to implement the processing handler. You can follow the README in the Github repository to integrate it into your codebase.
For all other use cases, the approach provided below using a sidecar is recommended.
In this case, the python library mentioned above is packaged into another docker image and runs alongside your main code in a separate container as a sidecar. You will need to write your processing code as HTTP service.
The tfy-async-sidecar component consumes items from the queue and calls your HTTP service with the payload as a POST request. The tfy-async sidecar code is open source and available here. It has adapters to consume the messages from the different types of queue as mentioned above. If you need support for additional queues please feel free to reach out, or contribute the queue adapter on this repository.
Once the sidecar pulls the message from the queue, it needs to deliver it to the HTTP service which is written by the user. There are no constraints on the HTTP service and you can write it using your own preferred framework like FastAPI, Flask, ExpressJS, etc. The service needs to expose an endpoint where it will accept the messages in the queue as the body in the POST request. You can provide that endpoint as an input in the async service deployment spec.
After the HTTP service receives the message, it processes it and returns the response to the sidecar. If we configure an output queue, the sidecar can then go and write the data back to the output queue. You can also handle the writing in your service code and not add the output queue configuration if you want.
The sidecar acks the message to the input queue once it has received the response from the HTTP service and written the response to the output queue. If there is a failure at any of the intermediate steps, the queue will redeliver the message to one of the replicas again for processing. This ensures that there is a high level of reliability for the input messages to be processed.
If you’re new to the TrueFoundry Async Service, follow the comprehensive guide given below on how to deploy your service as an asynchronous service. Please make sure you have deployed a simple service on TrueFoundry before embarking on this tutorial. You will need to have the following components ready before deploying an async service.
You can follow Deploy your first Async Service guide to understand the basics of deploying Async Service.
You can configure the settings further using the guides below: