ADF BC REST Query and SQL Nesting Control Solution

I will talk about expert mode View Object (with hand written SQL), this View Object is created based on SQL join. So, thats my use case for today example. I will describe issue related to generated SQL statement and give a hint how to solve it. This is in particular useful, if you want to expose complex VO (SQL with joins and calculating totals) over ADF BC REST service and then run queries against this REST resource.

Code is available on my GitHub repository.

Here is SQL join and expert mode VO (the one where you can modify SQL by hand):

This VO is exposed through ADF BC REST, I will not go through those details, you can find more info about it online. Once application is running, REST resource is accessible through GET. ADF BC REST syntax allows to pass query string along with REST request, here I'm filtering based on StreetAddress='ABC':

On backend this works OK by default and generates nested query (this is expected behaviour for expert mode VOs, all additional criteria clauses will be added through SQL wrapping). While such query executes just fine, this is not what we want in some use cases. If we calculate totals or average aggregated values in SQL, we don't want it to be wrapped:

To prevent SQL wrapping we can call ADF BC API method in VO constructor:

While probably this works with regular ADF BC, it doesn't work with criteria coming from ADF BC REST. SQL query is generated with two WHERE clauses, after query nesting was disabled:

Possible solution proposed by me - override executeQueryForCollection method, do some parsing and change second WHERE to be AND, apply changed query string and then execute super:

This trick helps and query is generated as we would expect, criteria added from ADF BC REST query call is appended at the end of WHERE clause:

Flow Navigation Menu Control in Oracle VBCS

Oracle VBCS allows us to build multiple flows within the application. This is great - this helps to split application logic into different smaller modules. Although VBCS doesn't offer (in the current version) declarative support to build menu structure to navigate between the flows. Luckily this requirement can be achieved in few simple steps, please read John Ceccarelli post - Adding a Navigation Bar to a VBCS Application. I thought to go through instructions listed by John and test it out, today post is based on this. In my next posts I will take a look how to replace navigation bar menu structure with something more advanced, for example - menu slider on the left.

I think VBCS have great potential as JavaScript declarative development IDE. I see many concepts are similar to other Oracle declarative development tools, e.g. Forms, Oracle ADF. VBCS runs Oracle JET, all you build in VBCS is Oracle JET. Oracle takes care upgrading Oracle JET version in VBCS, I have applied recent patch (by click of the button) and latest JET version is available within our VBCS environment:

Coming back to flows in VBCS. We can create as many flows we want. Each flow could be based on one or multiple fragments (HTML/JS modules). Here I have created three flows, each with single fragment:

We can select flow and this will bring us flow diagram, where we could have navigation implementation between flow elements/fragments:

Fragment - this is where UI part is done:

So thats about flows and fragments. For someone with ADF background, this sounds very similar to task flows and fragments. Next we should see how to implement flow navigation, to be able to select flow from the top menu. VBCS application comes with so called shell page. This page is top UI wrapper, which contains application name, logged user info, etc. Here we can implement top level menu, which would navigate through application flows:

There must be default flow, which is displayed once application is loaded. Default flow is set in settings of the shell page. Go to settings and choose default flow, dashboard-flow in my case:

Next we need to add JET component - navigation list to the shell page, to render menu UI. You can do it by drag and drop, but easier is to switch shell page to source view and add navigation list HTML portion manually (you can copy paste it from source code uploaded to GitHub, see link at the end of this post) - highlighted HTML will render menu bar to navigate between flows:

Initially you will notice error related to JET navigation list not recognised, we need to import it. Another error - selection listener is not found, we will implement it.

To import JET navigation list component, go to source implementation of the shell page and add oj-navigation-list in component imports section - this will solve issue with unknown navigation list entry:

To execute action in VBCS, we must create Action Chain. Crete Action Chain within shell page - navigateToPage:

We need input parameter - flow name, which want navigate to. Create variable in Action Chain - currentFlow:

Add action of type Navigate to Action Chain, this will trigger navigation logic:

Go to Action Chain source and add "page": "{{$variables.currentFlow}}" under actions. This will force navigation to the flow, which will be passed through parameter:

Finally we create navigation list selection event (within shell page), this event will trigger action chain created above and pass current flow ID. We must create custom event and its name should match even name defined in JET navigation list in HTML (see above):

Choose to create custom event (it didn't work for me in Chrome, only in Safari browser. VBCS bug?) and provide same name as in navigation list component listener:

Choose our navigation Action Chain to be triggered from this event:

Just a reminder, event is called from navigation list selection:

Event is passing flow ID from currently selected tab item:

On runtime, dashboard flow is loaded by default:

We can switch to Jobs, etc.:

Download exported (runnable only in VBCS) VBCS app from GitHub repo.

Oracle Offline Persistence Toolkit - Controlling Online Replay

Few months ago I had a post about Oracle Offline Persistence toolkit, which integrates well with Oracle JET (JavaScript toolkit from Oracle) - Oracle JET Offline Persistence Toolkit - Offline Update Handling. I'm back to this topic with sample application upgraded to JET 5.1 and offline toolkit upgraded to 1.1.5. In this post I will describe how to control online replay by filtering out some of the requests, to be excluded from replay.

Source code is available on GitHub. Below I describe changes and functionality in the latest commit.

To test online replay, go offline and execute some actions in the sample app - change few records and try to search by first name, also try to use page navigation buttons. You will be able to save changes in offline mode, but if this is your first time loading app and data from other pages wasn't fetch yet, then page navigation would not bring any new results in offline mode (make sure to load more records while online and then go offline):

In online replay manager, I'm filtering out GET requests intentionally. Once going online, I replay only PATCH requests. This is done mainly for a test, to learn how to control replay process. PATCH requests are executed during replay:

Printing out in the log, each GET request which was removed from replay loop:

Replay implementation (I would recommend to read Offline Persistence Toolkit usage doc for more info):

This code is executed, after transition to online status. Calling getSyncLog method from Sync Manager - returns a list of requests pending replay. Promise returns function with array of requests waiting for online replay. I have marked function to be async, this allows to implement sequential loop, where each GET request will be removed one by one in order. This is needed, since removeRequest from Sync Manager is executed in promise and loop would complete too late - after we pass execute replay phase. Read more about sequential loop implementation in JS, when promise is used - JavaScript - Method to Call Backend Logic in Sequential Loop. Once all GET requests are removed, we execute sync method, this will force all remaining requests in queue to be replayed.

Data Conflict Solution for ADF BC REST with Versioning

I would like to share sample solution for data conflict processing in ADF BC REST using versioning. When multiple users are editing concurrently the same data row - it is important to inform user before overriding changes already committed by another user. There are other approaches to implement data conflict control, you should evaluate if solution explained below is suitable for your use case, before applying it.

Sample code can be obtained from GitHub repository.

I'm using custom change indicator property, to evaluate if client data is expired. Change indicator value is sent to the client together with request data. PATCH request must include current client side change indicator value, if change indicator will match value in backend - PATCH is allowed, otherwise new change indicator will be returned to the client and response will be marked with 409 Conflict status code. Based on this, client could decide either to resubmit PATCH request with new change indicator and overwrite current data in DB or refresh client side data and try to submit changes later.

In this example - PATCH was executed with valid change indicator, response status is 200 OK. New change indicator value is returned to the client (it should be submitted for the next PATCH call for current row):

To test data change conflict, I would go directly to DB and change same record. Change indicator will be updated too:

Client doesn't know about change indicator update (data was changed by another user). Client will include currently known change indicator value and execute PATCH. This will result in 409 Conflict status. Backend returns latest change indicator value in the response:

Data wasn't updated, PATCH request was stopped on the backend:

Client knows latest change indicator value and can submit it again - this time successful (no one else changed data in the meantime):

Status 200 OK is returned, along with new change indicator value. Data is changed in DB as expected:

Backend implementation is not complex. You need DB trigger, which will get value from DB sequence and assign it for each changed row:

ADF BC REST includes change indicator attribute, it is marked with Refresh on Update support. This allows to get latest value assigned from DB trigger and return it to the client:

In doDML method we compare change indicator attribute value currently stored in DB and the one which comes from the client. If values do not match (client doesn't have the latest value) - update is not allowed:

When update is not allowed, we also must change HTTP response code to be 409 Conflict. This will allow to execute error callback on client side and take required action to process data conflict on the client. HTTP response code is set from custom ADF BC REST filter:

Text Classification with Deep Neural Network in TensorFlow - Simple Explanation

Text classification implementation with TensorFlow can be simple. One of the areas where text classification can be applied - chatbot text processing and intent resolution. I will describe step by step in this post, how to build TensorFlow model for text classification and how classification is done. Please refer to my previous post related to similar topic - Contextual Chatbot with TensorFlow, Node.js and Oracle JET - Steps How to Install and Get It Working. I would recommend to go through this great post about chatbot implementation - Contextual Chatbots with Tensorflow.

Complete source code is available in GitHub repo (refer to the steps described in the blog referenced above).

Text classification implementation:

Step 1: Preparing Data

• Tokenise patterns into array of words
• Lower case and stem all words. Example: Pharmacy = pharm. Attempt to represent related words 
• Create list of classes - intents
• Create list of documents - combination between list of patterns and list of intents

Python implementation:

Step 2: Preparing TensorFlow Input

• [X: [0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, ...N], Y: [0, 0, 1, 0, 0, 0, ...M]]
• [X: [0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, ...N], Y: [0, 0, 0, 1, 0, 0, ...M]]
• Array representing pattern with 0/1. N = vocabulary size. 1 when word position in vocabulary is matching word from pattern
• Array representing intent with 0/1. M = number of intents. 1 when intent position in list of intents/classes is matching current intent

Python implementation:

Step 3: Training Neural Network

• Use tflearn - deep learning library featuring a higher-level API for TensorFlow
• Define X input shape - equal to word vocabulary size
• Define two layers with 8 hidden neurones - optimal for text classification task (based on experiments)
• Define Y input shape - equal to number of intents
• Apply regression to find the best equation parameters
• Define Deep Neural Network model (DNN)
• Run model.fit to construct classification model. Provide X/Y inputs, number of epochs and batch size
• Per each epoch, multiple operations are executed to find optimal model parameters to classify future input converted to array of 0/1
• Batch size
• Smaller batch size requires less memory. Especially important for datasets with large vocabulary
• Typically networks train faster with smaller batches. Weights and network parameters are updated after each propagation
• The smaller the batch the less accurate estimate of the gradient (function which describes the data) could be

Python implementation:

Step 4: Initial Model Testing

• Tokenise input sentence - split it into array of words
• Create bag of words (array with 0/1) for the input sentence - array equal to the size of vocabulary, with 1 for each word found in input sentence
• Run model.predict with given bag of words array, this will return probability for each intent

Python implementation:

Step 5: Reuse Trained Model

• For better reusability, it is recommended to create separate TensorFlow notebook, to handle classification requests
• We can reuse previously created DNN model, by loading it with TensorFlow pickle

Python implementation:

Step 6: Text Classification

• Define REST interface, so that function will be accessible outside TensorFlow
• Convert incoming sentence into bag of words array and run model.predict
• Consider results with probability higher than 0.25 to filter noise
• Return multiple identified intents (if any), together with assigned probability

Python implementation:

Oracle VBCS - Pay As You Go Cloud Model Experience Explained

If you are considering starting using VBCS cloud service from Oracle, may be this post will be useful. I will share my experience with pay as you go model.

Two payment models are available:

1. Pay As You Go - good when accessing VBCS time to time. Can be terminated at any time
2. Monthly Flex - good when need to run VBCS 24/7. Requires commitment, can't be terminated at any time

When you create Oracle Cloud account, initially you will get 30 days free trial period. At the end of that period (or earlier), you can upgrade to billable plan. To upgrade, go to account management and choose to upgrade promotional offer - you will be given choice to go with Pay As You Go or Monthly Flex:

As soon as you upgrade to Pay As You Go, you will start seeing monthly usage amount in the dashboard. Also it shows hourly usage of VBCS instance, for the one you will be billed:

Click on monthly usage amount, you will see detail view per each service billing. When VBCS instance is stopped (in case of Pay As You Go) - you will be billed only for hardware storage (Compute Classic) - this is relatively very small amount:

There are two options, how you can create VBCS instance - either autonomous VBCS or customer managed VBCS. To be able to stop/start VBCS instance and avoid billing when instance is not used (in case of Pay As You Go) - make sure to go with customer managed VBCS. In this example, VBCS instance was used only for 1 hour and then it was stopped, it can be started again at anytime:

To manage VBCS instance, you would need to navigate to Oracle Cloud Stack UI. From here you can start stop both DB and VBCS in single action. It is not enough to stop VBCS, make sure to stop DB too, if you are not using it: