Building Bigger Applications with FastAPI

I share a few tips and tricks on how to build a clean and readable REST API for larger apps with FastAPI framework. You will learn how to split endpoints into separate Python scripts and assign tags with prefixes for all endpoints from the given router at once.

 

Web API with FastAPI, RabbitMQ and Celery

With Web API you can create access to microservice functionality. In this video, I explain how to create scalable Web API with FastAPI, Celery and RabbitMQ. Celery is responsible to execute async Web API requests and RabbitMQ enables the communication between Web API and microservices.

 

Oracle Offline Persistence Toolkit - Applying Server Changes

This is my final post related to Oracle Offline Persistence Toolkit. I will show simple example, which explains how to apply server changes, if data conflict comes up. Read previous post about - Oracle Offline Persistence Toolkit - Submitting Client Changes.

To apply server changes is easier, than to apply client changes. You need to remove failed request from sync queue and fetch server data to client by key.

Example of data conflict during sync:

User decides to cancel his changes and bring data from the server. GET is executed to fetch latest data and push it to the client:

In JS code, first of all we remove request from sync queue, in promise we read key value for that request and then refetch data:

Download sample code from GitHub repository.

Oracle Offline Persistence Toolkit - Submitting Client Changes

One of the key topics related to Oracle Offline Persistence toolkit - submitting client changes to backend when data conflict exists. If data was updated on the backend, while client was offline and client wants to submit his changes - we inform about the conflict and ask what client really wants to do. If client choose to submit changes, this means we should push client changes to the backend with the latest change indicator.

There is a special case, when client updates same data multiple times while offline - during online sync we need to make sure, change indicator will be retrieved in after sync and applied in before sync listeners, to make sure subsequent requests execute correctly. Check my previous post about before request sync listener - Oracle Offline Persistence Toolkit - Before Request Sync Listener.

Example - let's update a record and submit change to the backend:

Assume another user is offline and updates same record:

User updates same record again, before going online. Now we will have two requests in the sync queue:

Once going online, sync will be executed and we will get conflict for the first request (same row was updated already by another user). At this moment, after sync listener will get info about conflict and will cache latest change indicator value returned from backend. If user decides to apply his changes, requests is removed, new request is constructed with the latest change indicator value received from backend and this request is inserted into sync queue:

If same record was updated multiple times, second request will fail too - because this request wasn't updated yet with latest change indicator:

Assuming user decided to apply changes from the second request too, we will update request with latest change indicator and submit it for sync. In after sync listener, change indicator value stored in local cache will be updated.

Successful sync with change indicator = 296:

New change indicator value will be retrieved in after sync listener and applied in before sync listener for the second request, updating same data row:

Here is the code, which allows user to apply changes to backend. We remove failed request, update it and create new request in sync queue, resuming sync process:

Download sample code for the described use case from my GitHub repository.

Oracle Offline Persistence Toolkit - Before Request Sync Listener

One more post from me related to Oracle Offline Persistence Toolkit. I already described how after request listener could be useful to read response data after sync - Oracle Offline Persistence Toolkit - After Request Sync Listener. Today will explain when before request listener could be useful. Same as after request listener, it is defined during persistence manager registration:

Before request listener must return promise. We can control resolved action. For example if there is no need to update request, we simply return continue. We would need to update request, if same row is updated multiple times during sync. Change indicator value must be updated in request payload. We read latest change indicator value from array, initialised in after request listener. Request payload is converted to JSON, value updated and then we construct new request and resolve it with replay. API allows to provide new request, by replacing original:

Here is the use case. While offline - update value:

While remaining offline, update same value again:

We should trace executed requests during sync, when going online. First request, initiated by first change is using change indicator value 292:

Second request is using updated change indicator value 293:

Without before and after request listener logic, second request would execute with same change indicator value as the first one. This would lead to data conflict on backend.

Sample application code is available on GitHub.

Oracle Offline Persistence Toolkit - After Request Sync Listener

In my previous post, we learned how to handle replay conflict - Oracle Offline Persistence Toolkit - Reacting to Replay Conflict. Additional important thing to know - how to handle response from request which was replayed during sync (we are talking here about PATCH). It is not as obvious as handling response from direct REST call in callback (there is no callback for response which is synchronised later). You may think, why you would need to handle response, after successful sync. Well there could be multiple reasons - for instance you may read returned value and update value stored on the client.

Listener is registered in Persistence Manager configuration, by adding event listener of type syncRequest for given endpoint:

This is listener code. We are getting response, reading change indicator value (it was updated on the backend and new value is returned in response) and storing it locally on the client. Additionally we maintain array with mapping of change indicator value to updated row ID (in my next post I will explain why this is needed). After request listener must return promise:

On runtime - when request sync is executed, you should see in the log message printed, which shows new change indicator value:

Double check in payload, to make sure request was submitted with previous value:

Check response, you will see new value for change indicator (same as in after request listener):

Sample code can be downloaded from GitHub repository.

Oracle Offline Persistence Toolkit - Reacting to Replay Conflict

This is next post related to Oracle Offline Persistence Toolkit. Check my previous writing on same subject - Implementing Handle Patch Method in JET Offline Toolkit. Read more about toolkit on GitHub repo.

When application goes online, we call synchronisation method. If at least one of the requests fails, then synchronisation is stopped and error callback is invoked, where we can handle failure. In error callback, we check if failure is related to the conflict - then we open dialog, where user will decide what to do (to force client changes or take server changes). Reading latest change indicator value from response in error callback (to apply it, if user decides to force client changes in the next request):

Dialog is simple - it displays dynamic text for conflicted value and provides user with a choice of actions:

Let's see how it works.

User A editing value Lex and saving it to backend:

User B is offline, editing same value B and saving it in local storage:

We can check it in the log - changes value was stored in local storage:

When going online, pending requests logged offline, will be re-executed. Obviously above request will fail, because same value was changed by another user. Conflict will be reported:

PATCH operation fails with conflict code 409:

User will be asked - how to proceed. To apply changes and override changes in the backend, or on opposite take changes from the backend and bring them to the client:

I will explain how to implement these actions in my next post. In the meantime you can study complete application available on GitHub repo.

ADF BC REST Query and SQL Nesting Control Solution

I will talk about expert mode View Object (with hand written SQL), this View Object is created based on SQL join. So, thats my use case for today example. I will describe issue related to generated SQL statement and give a hint how to solve it. This is in particular useful, if you want to expose complex VO (SQL with joins and calculating totals) over ADF BC REST service and then run queries against this REST resource.

Code is available on my GitHub repository.

Here is SQL join and expert mode VO (the one where you can modify SQL by hand):

This VO is exposed through ADF BC REST, I will not go through those details, you can find more info about it online. Once application is running, REST resource is accessible through GET. ADF BC REST syntax allows to pass query string along with REST request, here I'm filtering based on StreetAddress='ABC':

On backend this works OK by default and generates nested query (this is expected behaviour for expert mode VOs, all additional criteria clauses will be added through SQL wrapping). While such query executes just fine, this is not what we want in some use cases. If we calculate totals or average aggregated values in SQL, we don't want it to be wrapped:

To prevent SQL wrapping we can call ADF BC API method in VO constructor:

While probably this works with regular ADF BC, it doesn't work with criteria coming from ADF BC REST. SQL query is generated with two WHERE clauses, after query nesting was disabled:

Possible solution proposed by me - override executeQueryForCollection method, do some parsing and change second WHERE to be AND, apply changed query string and then execute super:

This trick helps and query is generated as we would expect, criteria added from ADF BC REST query call is appended at the end of WHERE clause:

Oracle Offline Persistence Toolkit - Controlling Online Replay

Few months ago I had a post about Oracle Offline Persistence toolkit, which integrates well with Oracle JET (JavaScript toolkit from Oracle) - Oracle JET Offline Persistence Toolkit - Offline Update Handling. I'm back to this topic with sample application upgraded to JET 5.1 and offline toolkit upgraded to 1.1.5. In this post I will describe how to control online replay by filtering out some of the requests, to be excluded from replay.

Source code is available on GitHub. Below I describe changes and functionality in the latest commit.

To test online replay, go offline and execute some actions in the sample app - change few records and try to search by first name, also try to use page navigation buttons. You will be able to save changes in offline mode, but if this is your first time loading app and data from other pages wasn't fetch yet, then page navigation would not bring any new results in offline mode (make sure to load more records while online and then go offline):

In online replay manager, I'm filtering out GET requests intentionally. Once going online, I replay only PATCH requests. This is done mainly for a test, to learn how to control replay process. PATCH requests are executed during replay:

Printing out in the log, each GET request which was removed from replay loop:

Replay implementation (I would recommend to read Offline Persistence Toolkit usage doc for more info):

This code is executed, after transition to online status. Calling getSyncLog method from Sync Manager - returns a list of requests pending replay. Promise returns function with array of requests waiting for online replay. I have marked function to be async, this allows to implement sequential loop, where each GET request will be removed one by one in order. This is needed, since removeRequest from Sync Manager is executed in promise and loop would complete too late - after we pass execute replay phase. Read more about sequential loop implementation in JS, when promise is used - JavaScript - Method to Call Backend Logic in Sequential Loop. Once all GET requests are removed, we execute sync method, this will force all remaining requests in queue to be replayed.

Data Conflict Solution for ADF BC REST with Versioning

I would like to share sample solution for data conflict processing in ADF BC REST using versioning. When multiple users are editing concurrently the same data row - it is important to inform user before overriding changes already committed by another user. There are other approaches to implement data conflict control, you should evaluate if solution explained below is suitable for your use case, before applying it.

Sample code can be obtained from GitHub repository.

I'm using custom change indicator property, to evaluate if client data is expired. Change indicator value is sent to the client together with request data. PATCH request must include current client side change indicator value, if change indicator will match value in backend - PATCH is allowed, otherwise new change indicator will be returned to the client and response will be marked with 409 Conflict status code. Based on this, client could decide either to resubmit PATCH request with new change indicator and overwrite current data in DB or refresh client side data and try to submit changes later.

In this example - PATCH was executed with valid change indicator, response status is 200 OK. New change indicator value is returned to the client (it should be submitted for the next PATCH call for current row):

To test data change conflict, I would go directly to DB and change same record. Change indicator will be updated too:

Client doesn't know about change indicator update (data was changed by another user). Client will include currently known change indicator value and execute PATCH. This will result in 409 Conflict status. Backend returns latest change indicator value in the response:

Data wasn't updated, PATCH request was stopped on the backend:

Client knows latest change indicator value and can submit it again - this time successful (no one else changed data in the meantime):

Status 200 OK is returned, along with new change indicator value. Data is changed in DB as expected:

Backend implementation is not complex. You need DB trigger, which will get value from DB sequence and assign it for each changed row:

ADF BC REST includes change indicator attribute, it is marked with Refresh on Update support. This allows to get latest value assigned from DB trigger and return it to the client:

In doDML method we compare change indicator attribute value currently stored in DB and the one which comes from the client. If values do not match (client doesn't have the latest value) - update is not allowed:

When update is not allowed, we also must change HTTP response code to be 409 Conflict. This will allow to execute error callback on client side and take required action to process data conflict on the client. HTTP response code is set from custom ADF BC REST filter: