10) Abstraction Rule Calculation for Fraud Prevention

This Blog entry is from the Getting Started in Jube section in Learn Jube.

An Abstraction Calculation compares one Abstraction Rule value to another using reasonably simple arithmetic. For example, it is useful to take two values, dividing one by another to create a more powerful ratio.

The page is available by navigating through the menu as Entity Models > Abstraction and Fitting >>> Abstraction Calculation:

1.PNG

Clicking the required entity model in the tree towards the left hand side exposes the page for creating a new Abstraction Calculation:

2.PNG

The parameters available to the page are described in the following table:

Name

Description

This Example

Abstraction Calculation Type

The Abstraction Type as documented in the Abstraction Calculation Execution section of this document. For example Divide would cause a division of the Left hand Abstraction Rule Value by the Right hand Abstraction Rule Value.

Divide

Abstraction Left

The name of the Abstraction Rule to occupy the Left hand side of the arithmetic.

Refunds Sum

Abstraction Right

The name of the Abstraction Rule to occupy the Right hand side of the arithmetic.

Total Sales Sum

Complete the Abstraction Calculation a per the above table and as below:

3.PNG

Click the Add button to create a version of the Abstraction Deviation:

4.PNG

Allow the model a couple of moments to syncronise and then post an Account Financial Transaction using the HTTP Post Tool:

5.PNG

It can be seen that a new element block has been created in the response detailing the Calculations. In this case, the Abstraction has been multiplied by itself to create a square, although it is more commonly used in the creation of ratios using Divide.

9) Abstraction Deviation for for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

An Abstraction Deviation compares an Abstraction Rule value calculated for a transaction in process with what typically is observed against previous Abstraction Rules stored in the cache.

The page is available by navigating through the menu as Entity Models > Abstraction and Fitting >>> Abstraction Deviation:

1.PNG

Clicking on the Example Fraud Prevention model in the tree towards the left hand side starts the process of adding an Abstraction Deviation:

2.PNG

The parameters available to the Abstraction Deviation page as detailed in the following table:

Name

Description

This Example

Abstraction Deviation Type

The type of Abstraction Deviation to be calculated as documented in the Abstraction Deviation Rule Aggregation and Comparison section of this documentation.

Z Score

Abstraction

The Abstraction Rule name to to compare across historic values and the current value.

Velocity

Complete the page with the parameters that are set forth in the table above:

3.PNG

Clicking the Add button to create a version of the Abstraction Deviation:

4.PNG

Posting an Account Financial Transaction to the model:

5.PNG

It can be observed that a new element block called Deviation has been created which is comparing the current Abstraction with that of the history of that Abstraction.

8) Cross Model Abstraction Reference

This Blog entry is from the Getting Started in Jube section in Learn Jube.

The Cross Model Abstraction Page allows for the retrieval of Abstraction Rule values from other Entity Models. Cross Model Abstractions are a useful means of cross model communication alongside TTL Counters, which can also be incremented across models. A Cross Model Abstraction Value fetch is the process of retrieving the most current execution of a transaction in another model for that entity value. For example, if there are two models which share the same entity (e.g. an Account Identifier) yet they process inherently different data, the values created in Abstraction Rules in one model can be referenced in another model. Cross Model Abstraction Rules are one means of communicating between models with another possible means being TTL counters.

The page is available by navigating through the menu as Entity Models >>> Abstraction and Fitting > Cross Model Abstraction Reference. The page is presented as a tree objects for the purposes of administering a model child objects:

1.PNG

The general administration of this tree is as documented in the child object section of this documents, with the fields as defined follows:

Name

Description

This Example

Model

The name of the model that the required Cross Model Abstraction belongs to. The model will be integrated for the most recent Abstraction Rule values for the last transaction for the Entity Key.

Traffic

Abstraction

The Abstraction Rule to be returned from the most recent Abstraction for the Entity Key.

Is_Windows

If following the Blog entries step by step, there exists only one Entity Model at this time, hence there would not be a Entity Model available. For the purposes of demonstration, a second model titled Account Non Monetary Events has been created. The second model processes audit events from customer accounts, such as change of address:

2.PNG

With the following Request XPath configuration example:

3.PNG

The IP is specified as a Search Key. A simple Abstraction Rule has been created in the new model, which counts up all events on an IP:

4.PNG

For completeness, a some events have been processed through the new Entity Model, but that is outside the scope of this Blog Entry however, the AccountID element as the same value as contained in the Account Financial Transaction messages taken from the formats document (Rich16).

5.PNG

Return to the Cross Model Abstraction Reference and create an entry as per the parameters set out in the table above and the following:

6.PNG

Click the Add button to create a version of this Cross Model Abstraction Rule:

7.PNG

On recalling the Account Financial Transactions XML Post, it can be see that an entry has been created reaching into the value created in the new model:

8.PNG

It can be seen that the Abstraction Rule has been returned and merged as if it were originating from the same entity model.

7) Abstraction for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

The In Model Abstraction Rules page is where rules are created for the aggregation of data available in the Cache. The Abstractions Rules functionality is one of the most intrinsic processes in the Entity Model processing being responsible for the creations of aggregate statistics based on the content of the cache of data retained for that model. The abstraction rules are grouped together by their search keys.

A search key is a value that will be used as a predicate in retrieval of data for the subsequent execution of Abstraction rules that use that search key. For example, if a transaction is being processed through a model, if an account identifier is configured to be a search key, a selection of all records in the model cache will be returned where that account identifier matches (put simply it will return all transactions for that customer account). Search keys are defined in the Request XPath definition and there can be many search keys - thus cache database queries - per transaction being executed through the model. The process is different when a Search Key is defined to be retrieved from a cache, as these values are pre-calculated by the Search Key Cache thread, documented extensively in its own right.

Firstly, Search Keys need to be set up, in this example IP, by revisiting the Request XPath page via Entity Models >>> References >>> Request XPath:

20.PNG

Expand on the mode tree on the left hand side to expose all Request XPath for this model, then click on IP:

2.PNG

For the Request XPath, click the checkbox titled Search Key:

3.PNG

With Search Key having been checked, click on the Update button towards the base of the page:

4.PNG

With a search key now being available, navigate through the menu as Entity Models >>> Abstraction and Fitting >>> In Model Abstraction. The page is presented as a tree objects for the purposes of administering a model child objects:

5.PNG

Click on the model towards the left hand side to add an Abstraction Rule:

6.PNG

The Page exists to create Abstraction Rules which are code fragments will will be compiled into the Jube Engine. Abstraction Rules accept data and TTL Counter objects and tests the code fragment for equality, returning a boolean flag to that effect. The Rule and Code Constructor Comprises the following:

Name

Description

Data

The fields created in the Request XPath page and Inline Scripts (if allocated).

TTL Counters

The TLL Counter and their current incremented values.

The page accepts the following parameters in the construction of Abstraction Rules:

Name

Description

This Example

Scope

The scope whether the rule is to be executed on a list of records having been returned from the Cache or only upon the transaction currently being processed through the model. None would imply that the scope of the Abstraction Rule is only the current transaction being processed and if it matches, then a 1 will be returned, otherwise zero (keeping in mind that Neural Networks will seek to weight it, hence the numeric values). If Entity Search would imply that a retrieval from the Mongo Datbase need be taken place and the Abstraction Rule is going to be tested against each transaction returned.

Entity Search

Search Key

The Search Key is the basis for several data retrievals to be made of the Mongo Database. For example, if a Search Key is specified to be an IP, then it would imply that a select is going to be made of the Mongo Database for all transaction matching on the IP of the current transaction thereafter applying the Abstraction Rule on each of the records to test for a match.

IP

Search Interval

The Search Interval is the time threshold reaching back from the Reference Date for the transaction being processed whereby only transactions exceeding this threshold are eligible for matching. For example, if a Search Interval is configured for 1 Day (taking the Search Interval and Search Value together) then this would imply that only records greater than the Reference Date minus One Day is eligible. It is worthy of note that the Search Interval and Search Value are not used in the recall from the Mongo Database, that will still return everything matching on the key up to the fetch limit specified in the model definition. The Search Interval is taken together with the Search Value.

d

Search Value

As Search Interval, with the Search value being comprising the length of time to reach back from the Reference Date.

1

Offset Type

All data matching on a search key is retrieved from the cache. The Offset type helps to select data from a subset of these records. This can be helpful if analysis need be done only on the last x records, for example. In most cases it will be set to none.

None

Offset Interval

The number of intervals to be offset by, for example the last x number of records returned from the cache.

3

Function Type

The process of aggregating the matches is different from the process of testing and matching the Abstraction Rules itself. The process of aggregation takes place after all matching has take place and the transaction matches for the rule have been set aside. The Function Type is the aggregation type to take place and is extensively documented in the Abstraction Function Definitions of this document. For example, if a Sum function is specified, then the function will look to count up records using the Function Key field (e.g. Transaction Amount).

Sum

Function Key

The Function Key is available only on certain Function Types and represents the fields inside the transaction that are to be used for aggregation. For example, if the Sum Function Type is specified and the Function Key is specified as the Transaction Amount, the aggregation will seek to create a Sum for all of the Transaction Amounts. In the case of a Count, no Function Key would be available, as the Function Type would seek to count up the matches only.

CurrencyAmount

Complete the page with the parameters as prescribed above, setting a rule to match on everything:

7.PNG

Click the Add button to create a version of this Abstraction Rule:

8.PNG

Allow the platform a couple of moments to syncronise then post an Account Financial Transaction:

9.PNG

It can be seen that the sum of all events in the last day has been aggregated, and this will increase on each event. This value is available for subsequent rules.

6) TTL Counters for for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

Time to Live Counters are an alternative way to create model memory without needing to store the full transaction data in the Cache. TTL Counters are incremented as the result of an Activation Rule matching and are set to live for only a prescribed amount of time upon having been created. TTL Counters allow for a condition only counter, storing a lightweight counter entry and not the entire transaction picture. It follows that the storage of TTL Counters is substantially less than maintaining the cache.

Upon a TTL Counter Definition having been created it will be available in the Activation Rules page such that it can be incremented as a consequence of an Activation Rule matching.

The page is available by navigating through the menu as Entity Models >>> Abstraction and Fitting >>> TTL Counters. The page is presented as a tree objects for the purposes of administering a model child objects:

1.PNG

Click on the model tree entry towards the left hand side to add a new TTL Counter:

2.PNG

The fields available to this form are described in the following table:

Name

Description

This Example

TTL Counter Interval Type

The amount of time before the TTL Counter is to be decremented upon a counter entry having been created and incrementing the counter. The TTL Counter Interval Type is taken together with the TTL Counter Interval Value.

d

TTL Counter Interval Value

The amount of time before the TTL Counter is to be decremented upon a counter entry having been created and incrementing the counter. The TTL Counter Interval Type is taken together with the TTL Counter Interval Value.

1

Complete the page fields as set forth in the table above:

3.PNG

Click the add button to create a version of this TTL Counter:

4.PNG

Allow the platform a couple of moments to synchronise the configuration.

Mindful that a TTL Counter must be incremented by an Actvation Rule match, hence when recalled at this time would return zero, proceed to post the Account Financial Transaction XML:

5.PNG

It can be seen that TTL Counters are being recalled and aggregated and are available for subsequent rule processing.

5) Lists for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

The Lists Page is facilitates the creation of lists of data that can be referenced as an operator in all rules recalled in the Entity subsystem. For example, in Fraud, the Lists page can be used to maintain a list of bad customers which can be referenced using an 'is in list' or 'is not in list' operator in the Rule and Code Constructor. Lists are a good way to create Deny and Accept lists without having to embed large amounts of complexity into the Rule and Code Constrictor.

The page is available by navigating through the menu as Entity Models > References > Lists. The page is presented as a tree objects for the purposes of administering a model child objects:

1.PNG

To create a list, click on the Entity Model in the tree towards the left hand side:

2.PNG

Creating a list by the name of Bad_Customers:

3.PNG

Click the Add button to confirm creation of the List:

4.PNG

In the following example, the Bad_Customer list is now available for comparison in the Gateway Rule using the 'is in list' operator:

5.png

Upon creation of the List it can be seen that the child values of this list can be administered. The list values are maintained using a List control facilitating the addition, updating and removal of List Values:

6.PNG

To add a value to the list, a button is implemented titled 'Add Value':

7.PNG

Clicking the Add button will overlay a form and create a new entry in the List control:

8.PNG

Enter the required value, in this case a Bad Customer AccountID:

9.PNG

To commit the new value, click on the tick icon:

10.PNG

Or to cancel the creation of the Value click the opposing icon:

11.PNG

Upon the successful commit of the Value, it will appear in the List control:

12.PNG

To update a List Value, click on the pen icon for the list value to be updated:

13.PNG

Upon clicking the Pen icon, a form with a text box will be over lay:

12.PNG

Update the text box with the new value:

14.PNG

Commit by clicking on the Tick icon:

10.PNG

Or to cancel the creation of the Value click the opposing icon:

11.PNG

Upon the successful commit of the Value, it will appear updated in the List control:

15.PNG

To delete a List value entry, it is a simple matter of clicking on the Disk Icon next to the List Value targeted for deletion:

16.PNG

A confirmation of the delete will be sought:

17.PNG

Upon confirming the delete, it will be removed from the list of values:

18.PNG

4) Gateway Rules for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

In anticipation for this Blog entry, the all elements in the Account Financial Transactions XML document has been represented as a Request XPath:

1.PNG

All Request XPath has been configured as Response Payload hence will be returned in the HTTP response:

2.PNG

Gateway Rules are pre filtering and sampling to reduce the transaction traffic volume for the processing of intensive model functions. Transactions will only proceed to subsequent steps in model processing in the event that Gateway Rule matches. Upon a Gateway Rule matching, it will be tested against a random sample as another means to reduce the volume. The random sample at the Gateway Rule level is used quite often in Advertising Technology where there may be an abundance of advertising impressions available yet it is not practical to acquire all, hence processing need only take place upon a smaller yet representative sample.

The page is available by navigating through the menu as Entity Models > References > Gateway Rules. The page is presented as a tree objects for the purposes of administering a model child objects:

3.PNG

The Page exists to create Gateway Rules which are code fragments will will be compiled into the Jube Engine. Gateway Rules accept data objects and tests the code fragment for equality, returning a boolean flag to that effect. The Rule and Code Constructor Comprises the following data Objects:

Name

Description

Data

The fields created in the Request XPath page and Inline Scripts (if allocated).

To add a new Gateway rule, click on the Entity Model in the tree towards the left hand side:

4.PNG

The parameters for the Gateway Rule are described as follows:

Name

Description

This Example

Gateway Sample

A percentage value, between 0 and 1, detailing the sample that should be allowed through to subsequent processing. The sample will only be taken in the event that the Rule returns as true. For example, if the Gateway Sample is set to .2, it would imply that only 20% of the matching transactions will be passed through for subsequent processing. It is extremely useful in Advertising Technology where there is an extremely large volume of impressions yet it is not practical for each impression to be acquired, hence a representative sample of the large universe is adequate.

1 (for 100%)

Max Response Elevation

Upon a Gateway Rule Matching, the Maximum Response Elevation is set such to provide some risk management to the magnitude of the response elevation. For example, if the Response Elevation is used to instruct a bid, the open end of this could be cause for alarm. This value is the largest response elevation that may be responded, with any values exceeding this value being truncated to this value. Keep in mind that there are similar limits at the model level also.

11.0000

Complete the parameters for the Gateway Rule as the aforementioned parameters and as below:

5.PNG

Only in the event that a Gateway rule matches will processing continue. If no Gateway rules match, the response message will simply be sent back upon termination of processing:

6.PNG

The the event that no subsequent processing has taken place, yet there is configuration in place, it is most likely that an event has failed to match upon a Gateway Rule.

3) Inline Scripts Allocation for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

The Entity Models inside Jube are quite extensible via the use of Inline Scripts, which are documented comprehensively in a separate section of this document. Inline Scipts once registered in the database are available for all Entity Models, however they must be registered in a model. The Inline Script page is to facilitate the linking of Inline Scripts registered in the Jube platform and this model.

The page is available by navigating through the menu as Entity Models > References > Inline Scripts. The page is presented as a tree objects for the purposes of administering a model child objects:

1.PNG

Add a new Inline Script by clicking an Entity Model entry in the tree towards the left hand side:

2.PNG

The parameters available to Inline Script allocation are described in the following table:

Name

Description

This Example

Inline Script

The Inline Script to be invoked on each request being made to the Entity Model. Inline Scripts are documented extensively in another section of this documentation and this functionality serves to link that Inline Script with a model. In this example, the model has been linked with an Inline Script responsible for taking an IP address and looking up the geographic location of the IP address.

IP

Complete the Inline Script page as the above table of parameters:

3.PNG

Click Add to create the first version of this Inline Script allocation in the Entity Model:

4.PNG

Allow the platform a few moments to syncronise the Inline Script Allocation across all of the nodes, then repeat the HTTP Post as described in previous Blog Entries using Account Financial Transactions.

5.PNG

Any properties that have been specified as Response Payload will be written out, more importantly all properties will be available for the creation of Gateway Rules, as dealt with in subsequent Blog entries.

2) Request XPath for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

In previous Blog Entries and Entity Model was configured to consume XML and extract some important parameters from the XML body (but it could also be JSON).

When a request is received from a HTTP Endpoint for a model it will have data contained within either the Querystring or more likely the POST body of the request. Depending on whether the Message Type has been configured to be XML or JSON will determine the parsing algorithm to be applied to it. Once parsed data can be extracted from using either XPath or JSON Path. The Request XPath page is where model fields are specified alongside their position in the POST Body or the Querystring. Various other parameters specifying how this field should be processed by the model are included also.

The page is available by navigating through the menu as Entity Models > References > Request XPath. The page is presented as a tree objects for the purposes of administering a model child objects:

1.PNG

The general administration of this tree is as documented in the child section of this Blog, with the fields as defined follows:

Name

Description

This Example

Priority

The processing priority of the Request XPath which is useful if there are two fields of the same name, but looking at different elements in the payload.

0

Hash Entity Key Composite

A flag that specifies the field to be used in composite with other fields flagged in the same way. During transaction processing all fields which are flagged as such will be concatenated together before being hashed using MD5. This hashed value will become, hence overwrite, the value extracted as per the model definition. It is useful in the absence of a guaranteed unique value in the transaction yet also where storage of the data (i.e. PII) has regulatory restriction.

False

Hash Entry Key Composite

A flag that specifies the field to be used in composite with other fields flagged in the same way. During transaction processing all fields which are flagged as such will be concatenated together before being hashed using MD5. This hashed value will become, hence overwrite, the value extracted as per the model definition. It is useful in the absence of a guaranteed unique value in the transaction yet also where storage of the data (i.e. PII) has regulatory restriction.

False

Hash Entry Key Composite Sequence

Reserved for future use.

False

Hash Entity Key Composite Sequence

Reserved for future use.

False

Payload Location

A switch that specifies the location of the data for this field. If Body is specified then the Xpath field will be targeted at the parsed POST body whereas if Querystring is specified the XPath field will expect only the name of the Querystring value.

Body

XPath

Depending upon whether the Querystring of Body is targeted, the location of the data to be extracted. In the event that Querystring is specified as the Payload Location then it will require the name of that Querystring value. In the event that Body is specified as the Payload location then it will require XPath or JSON Path depending upon the Message Type.

//IP

XPath Expression

An expression has special meaning in the XPath language and allows for calculations to be performed in XPath (for example creating an average of several fields). This flag specifies if the XPath is an expression rather than a selector, allowing it to deal with the slightly different return type.

False

Search Key

If the field is flagged as a search key it allows the value extracted to be used to query the cache in the Abstraction Rule processing. For example, if a count of all transactions having taken place on an IP address are required, this would be flagged as a Search Key which would mean that during execution the value extracted in this process will be used as a predicate in a select from the Cache (i.e.All from Cache where Cache IP = This IP).

False

Cache

For a Search Key to retrieve data for each transaction it can place significant load on the Mongo Database and Engine and is quite impractical if there might be extremely large amounts of data available for that Search Key (e.g. consider a Merchant with millions of transactions a day). A Search Key Cache defers the processing of these Search Keys to a background engine to process each of the possible Search Keys, saving the calculations off to a Cache, with that Cache being referenced real-time rather than the raw data.

NA

Cache Key Interval Type

The Cache Key Interval Type is the date interval taken together with the Cache Key Interval Value which specifies how frequently the Cache Keys should be recalculated (but even then, it is only done on new data).

NA

Cache Key Interval Value

The Cache Key Interval Value is the date interval taken together with the Cache Key Interval Type which specifies how frequently the Cache Keys should be recalculated (but even then, it is only done on new data).

NA

Cache Key Sample

In order to reduce processing demand in the calculation of Search Keys, a representative sample can be specified to reduce the number of records brought back for each possible search key. All distinct search keys will be processed but only a sample of data for that search key value will be returned. In most cases a sizeable representative sample is perfectly adequate.

NA

Fetch Limit

For each distinct search key value, the maximum number of transactions that can be returned from the Mongo Database for calculations to be performed.

NA

Cache Key TTL Interval Type

The Cache Key TTL Interval Type is the date interval taken together with Cache Key TTL Interval Value and is how long a distinct Search Key, once calculated, should live before being purged from the Cache.

NA

Cache Key TTL Interval Value

The Cache Key TTL Interval Value is the date interval taken together with Cache Key TTL Interval Type and is how long a distinct Search Key, once calculated, should live before being purged from the Cache.

NA

Data Type

The data type of the data being extracted from the HTTP Request. The data type specified is important as it effects the types of predication that can take place in the rules. Possible values are:

String: Text and General Data

Integer: Numbers without any decimal places

Float: Numbers with decimal places.

Date: Date and Time.

Boolean: True or False

String

Encrypt At Rest

If the encrypt at rest flag is specified it means that the data for this field will be encrypted using an AES key stored in the Engines configuration file just prior to it being committed to the database. Upon retrieval of the data, the field will be decrypted such that its contents are available in memory.

Please check with support that this option is enabled.

False

To add a new Request XPath, click on the Entity Model recently created in the top of the tree to the left hand side:

2.PNG

Complete the fields with the value as set forth in the table above, being certain to set the Response Payload flag to ensure that the extract value is returned for testing:

3.PNG

Clicking Add to create the first version of the Request XPath for this Entity Model:

4.PNG

Allow the platform a few moments to syncronise the new Request XPath across all of the nodes, then repeat the HTTP Post as described in previous Blog Entries using Account Financial Transactions.

5.PNG

Notice how the IP is present in the response payload which would indicate that the IP was extracted from the request, processed and then included in the response payload.

This Blog entry should be repeated for each element required in processing and available in the XML POST.

In the event that the request payload if of JSON format, the approach would be the same except in place of XPath, JSON path would be used.

1) Creating an Entity Model for Fraud Prevention using Account Financial Transactions.

This Blog entry is from the Getting Started in Jube section in Learn Jube.

In this Blog entry an Entity model will be created to consume the Account Financial Transactions format with the objective of creating a fraud management platform.

Before proceeding with this Blog post, review the XML for the Account Financial Transaction in the formats document available at:

http://download.jube.io/formats.pdf

It can be seen that the XML is comprised of many elements that make up a financial transaction such as Amount, Location, Authentication Type etc. These elements can be added to in subsequent procedures and most will not be absolutely required:

1.PNG

Before embarking on this procedure, be sure to have read the Blog entries on messaging and integration and how to use the Postman page in Jube.

Fraud Prevention would be an Entity Model as it is classification rather than numeric prediction, in so far as the goal of the model is to classify a transaction as being either fraudulent of genuine.

Start by navigating to Entity Models >>> Entity Models

2.PNG

The Entity Models screen is where a model is first defined and allocated an EntityAnalysisModelGUID for model recall. Start by clicking on the Add button to expose the models basic parameters:

3.PNG

The parameters required for the creation of a new Entity Model will be displayed:

4.PNG

The tables as follows gives an introduction to each parameter available in Entity Model creation, excluding those explained in the Blog entries dealing with general user interface concepts:

Name

Description

This Example

GUID

When an Entity Model is created it is allocated a GUID (Globally Unique Identifier) which will be the key to the model upon it being recalled via the HTTP Endpoint. The recall of Entity Models is extensively documented in the HTTP Endpoint Request Payload and Switching section of this document. For example, the GUID is used in recalling the model via this endpoint as:

http://api.jube.io:5555?EntityAnalysisModelGUID=6b02e83e-f40f-4816-ad95-1329437b831b

TDB on Submission of new Entity Model

Message Type

The HTTP Request will likely contain a POST body (although it can accept data from only the Querystring). This parameter specifies if that HTTP POST Body is of XML or JSON format.

The parameter will be used to specify which parsing algorithm and schema to use, then whether the XPath parameters represent XML XPath or JSON JSONPath.

There is not much difference in parse or extraction performance between XML and JSON.

XML

Entity Name

The name of the field, as extracted from the HTTP POST Body or Querystring, that represents the entity identifier or key. An example of an Entity might be an Account identifier in the processing of financial transactions.

AccountID

Entity Payload Location

The Entity can be extracted either from the HTTP POST Body or the Querystring of the HTTP Request. This parameter specifies where the data is to be extracted from.

Body

Entity XPath

In the event that Body is specified for the Payload Location, the XPath or JSON Path specifying the location of the Entity in the HTTP Post Body. In the event that Querystring is specified for the Payload Location, the name of the Querystring as the key to the value collection in the list of Querystring values contained within the HTTP Request URL.

//AccountID

Entry Name

The name of the field, as extracted from the HTTP POST Body or Querystring, that represents the entry identifier or key. An example of an Entry might be a Transaction identifier in the processing of financial transactions.

TxnID

Entry Payload Location

The Entity can be extracted either from the HTTP POST Body or the Querystring of the HTTP Request. This parameter specifies where the data is to be extracted from.

Body

Entry XPath

In the event that Body is specified for the Payload Location, the XPath or JSON Path specifying the location of the Entity in the HTTP Post Body. In the event that Querystring is specified for the Payload Location, the name of the Querystring as the key to the value collection in the list of Querystring value contained within the HTTP Request URL.

//TxnID

Reference Date Name

The name of the field, as extracted from the HTTP POST Body, Querystring or using the current time, that represents the Reference Date. An example of a Reference Date might be the date time as of Now taken from the API endpoints system time.

TxnDateTime

Reference Date Payload Location

The Reference Date can be extracted either from the HTTP POST Body, the Querystring, the HTTP Post Request or taken from the system time of the API endpoint. This parameter specifies where the data is to be extracted from.

Body

Reference Date XPath

Subject to Now not having been specified for the Payload Location, the XPath or JSON Path specifying the location of the Reference Date in the HTTP Post Body. In the event that Querystring is specified for the Payload Location, the name of the Querystring as the key to the value collection in the list of Querystring value contained within the HTTP Request URL.

//TxnDateTime

Cache TTL Interval

When data is processed through an Entity Model it relies a subset or fast-moving data residing in a Mongo Database, known as the Cache. The interval is the time denomination which is used to specify the maximum amount of time that data should be resident in the cache before being purged and is taken together with Cache TTL Value.

Days

Cache TTL Value

When data is processed through an Entity Model it relies a subset or fast-moving data residing in a Mongo Database, known as the Cache. The interval is the number of intervals which is used to specify the maximum amount of time that data should be resident in the cache before being purged and is taken together with Cache TTL Interval.

30

Cache Fetch Limit

A limit applied on the retrieval of data from the cache set in order to avoid the real-time transaction processing being flooded by a very active entities (i.e. Accounts with an implausibly high number of transactions, such as test accounts). When retrieval is made from the Cache it is done so by selecting all records where there is a match on the Entity.

100

Enable Cache Entity

A toggle flag declaring if the model is to store events in the cache at all. It may be that the data volume is of such high volume that it is impractical to store and recall history data without needing to scale disproportionately.

True

Enable Cache Abstraction

After all processing has taken place, all calculations that were created as the result of processing are also stored in the cache to support the Abstraction Deviation functionality. This toggle flag declares the model capable of storing Abstraction in cache.

True

Enable Cache TTL Counters

TTL Counters are an alternative to storing all of the data in cache, instead only storing counter entries matching certain criteria. This toggle flag declares the model capable of storing TTL Counters.

True

Archive TTL Interval

The Archive relates to the storage of slow-moving data in support of analytics in the SQL Server database. The interval is the time denomination which is used to specify the maximum amount of time that data should be resident in the SQL Server Database before being purged and is taken together with Archive TTL Interval.

Days

Archive TTL Value

The Archive relates to the storage of slow-moving data in support of analytics in the SQL Server database. The value is the number of intervals which is used to specify the maximum amount of time that data should be resident in the cache before being purged and is taken together with Archive TTL Interval.

6

Mas Response Elevation

A value to truncate Response Elevations that are set to be higher than this value. It is commonly used in advertising technology to avoid over bidding.

11

Placing each one of the example values into their respective fields in the form:

1.PNG

Click the tick icon to commit the new Entity Model specification:

5.PNG

Confirmation that the Entity Model has been created will return in the form of a GUID having being created:

2.PNG

Allow the platform a few minutes to syncronise the new Entity Model with all of the endpoints, then navigate to the Postman page via Administration >>> HTTP Post Tools >>> Postman. Using the GUID, create a POST using the Account Financial Transaction XML to the https://api.jube.api endpoint.

Postman Paramater

Value

Format

XML

Querystring Pair Value Collection

EntityAnslysisModelGUID= f994ba45-1ef9-4af2-9164-870423e8e9d1

Request

Account Financial Transactions Taken from Formats document.

3.PNG

If Postman returns an XML document, echo back of several variables, it can be determined that Entity Model has been created and is now listening. The task now is to specify more elements to be extracted from the XML payload in the Request XPath page and as covered in subsequent Blog entries.

7) Code Construction Widget

This Blog entry is from the Getting Started in Jube section in Learn Jube.

Several pages in the platform rely on the construction of code and in all cases this is facilitated by a point and click construction tool or direct freehand code (yet heavily restricted with only certain language functions being allowed). In most cases the code is of vb.net dialect, unless it is used for reporting or Adaptation training in which case it is an SQL fragment.

The following example is taken from the Entity Models Activation Rules page, which can be navigated to via Entity Models >>> Activation >>> Activation Rules, then selecting any child:

1.PNG

The default is Builder which uses a point and click interface to build up the vb.net code.

The Rule Constructor takes the form of Object, Field, Comparison and Value. Depending upon the Rule that is being constructed, different Objects will be available. Hovering over the Data Object, there are a large amount of fields available for selection:

2.png

Upon selecting an Object field, the Comparison operator can be specified. The operators differ depending the area of the platform and how the model is configured:

3.png

Finally, the Value can be specified for comparison:

4.png

Upon each update made to the Rule and Code Constructor, the code fragment is written out to the code window:

5.PNG

Having the code written out in this manner is rather easier to review and also helps to identify if there are any obvious issues that would leads to compilation issues. Toggling to Coder, using the radio buttons towards the top of the the Code Construction Widget, it becomes possible to create code by hand:

6.PNG

When typing in code, every second and on change, the code is sent to the server to be parsed, returning any errors that may exist at that point of code construction:

7.PNG

It is incumbent on the user to continue constructing and correcting the code, until it is declared to be ‘Parsed and Compiled’:

8.PNG

Only a small number of tokens from the vb.net language are allowed in the Coder for security reasons.

As aforementioned, the Code Constriction Widget mostly creates vb.net dialect code, with the exception of the Adaptation page, which can be navigated to via Entity Models >>> Adaptation and Prescription, with an arbitrary child selected as follows:

9.PNG

In the case of Adaptation, the code is an SQL fragment which helps build a data set that is intended for machine learning:

10.PNG

It is not possible to update the Adaptation with hand crafted code.

6) Response Payload

This Blog entry is from the Getting Started in Jube section in Learn Jube.

All children have the option to specify if it should be specified as being a Response Payload. A Response Payload means that the content or value created by the object will be returned to the HTTP response stream, otherwise excluded. It is a means to reduce the amount of bandwidth that is used for extremely large models and thus speed up transport of the response message:

1.PNG

Irrespective of the Response Payload being selected, the value will still be stored in the archive database.

5) Name and Description

This Blog entry is from the Getting Started in Jube section in Learn Jube.

Each parent and child object entry will have a Name field, which is a character string accepting a maximum of 50 characters. A description field appears on a more irregular basis and is a character string accepting a large number of characters.

1.PNG

The Names for the object should always be unique for that object, for example each Abstraction Rule should have a unique name, whereas the description is of freehand.

4) Locked and Active

This Blog entry is from the Getting Started in Jube section in Learn Jube.

For every parent and for every child there exists an Active and Locked flag:

1.PNG

Both the Active and the Locked flags form part of the record and are updated in the same manner as the rest of its properties.

The Active flag provides a means to deactivate an object without having to delete it. In most cases deactivating a child object has a very similar behavior to deleting it, insofar as the entry will be unavailable for processing.

The Locked flag may be toggled in just one direction, Locking a record. If a record is Locked it means that all subsequent editing of that record is completely barred and the records is presented a read only. Locking records is useful to avoid the inadvertent update or deletion of important records. An example of objects that once created should not be deleted are Abstraction Rules which are in use my an Adaptation, as the alteration of these Abstraction Rules would render the results of the Adaptation inaccurate. Another example is ensuring that the Request XPath configuration, once created, remains unchanged such is the importance of these values in maintaining real time interfaces into the Jube Platform:

2.PNG

In the above example, note how the Update and Delete buttons are no longer available.

3) Child Versioning

This Blog entry is from the Getting Started in Jube section in Learn Jube.

For children there is a version system in place. When a record is created for a child object, it is always accompanied by version data:

1.PNG

The fields that relate to version of children are descried in the table as follows:

Name

Description

Version

The version in a sequential number that indicates the extent to which this record has been updated. Upon a record being created, it is allocated version 1, then for each update (i.e. click of the update button) the version will increment by one.

User Created

The name of the user that last created or updated this child record.

Created Date

The date and time this record was last created or updated.

For children all update and deletes are done on a logical basis, which means that they are marked as deleted in the database but still remain there for the purposes of audit. When a child record is updated, in actuality the record being updated is logically deleted and a new record with the incremented version number is created. It follows that there is record for each update maintained in the database.

2) Children

This Blog entry is from the Getting Started in Jube section in Learn Jube.

Children roll up to the Parents and they are administered via a common user interface spanning the platform. In this example, a Request XPath will be added to an Entity Model (whereby the Entity Model was used as an example of a Parent object previously). The Request XPath page is available by navigating to Entity Models >>> References >>> Request XPath:

1.PNG

Navigation around children takes place via a tree control, where each parent is represented as the upper level of the tree object. In the following example, it can be seen that each parent is the top level of the tree:

2.PNG

To expand on the top most level to expose children, simply click on the triangle icon next to the parent name:

3.PNG

On clicking the triangle icon, a list of all children rolling up to the parent will be revealed:

4.PNG

To add a child object, click on the parent entry in the tree for which you wish to create the child, exposing a blank for for the creation of a child object in the right hand pane:

5.PNG

Upon completing each field required of the child object in right hand panel, clicking the Add button at the base of the form will commit the object:

6.PNG

Confirmation of the object having been committed will be returned care of the Add button disappearing and being replaced with an Update and Delete button, alongside confirmation of the version and created date:

7.PNG

If the tree is expanded, the new entry will be added to that tree, otherwise the new entry can be viewed by expanding as aforementioned:

8.PNG

To update a child object, start by expanding on the tree as described in the Navigate subsection of this section of the documentation, then click on the child entry to be edited:

9.PNG

Alter the values of the child object as required, then click the Update button to commit the changes:

10.PNG

Upon clicking Update, confirmation of a change to the child object is received by the Version being incremented by one:

11.PNG

As with update, to delete a child object, start by expanding on the tree as described in the Navigate subsection of this section of the documentation, then click on the child entry to be deleted. Upon the child object being expanded into the right hand pane, click on the Delete button to confirm that this child object requires deletion:

12.PNG

Upon clicking Delete, a prompt will be displayed which will ask for confirmation of this delete:

13.PNG

Upon confirmation being given, the delete will take place. The entry on the left hand pane will be removed from the tree if it is still expanded, whereas the right hand pane will write out further confirmation of the delete having taken place:

13.PNG

1) Parents

This Blog entry is from the Getting Started in Jube section in Learn Jube.

There are several basic hierarchies created in the JUbe platform, where a parent is created and many configuration elements cascade down from then.

For the time being, knowing the parents in the Jube platform and how to navigate to them is adequate. The following table details how to navigate to the parent objects in Jube:

Parent

Navigation

Entity Models

Navigate to Entity Models >>> Entity Models

Symbols

Navigate to Symbol Models >>> Symbols

Cases Workflows

Navigate to Administration >>> Cases >>> Cases Workflows

Roles

Navigate to Administration >>> Roles

The parents share similar characteristics for the purposes of administration. Firstly, a list control will return all parents configured for that part of the platform. In the following example, all Entity Models are returned:

1.PNG

To move from one record to the next, use the scroll buttons or index buttons at the base of the list:

2.PNG

To Add a record to an object, a button is implemented at the top of the List Control:

3.PNG

Clicking on this button will overlay an entry form to accept a new record:

4.PNG

Upon completing the form with valid values for this object (each of these is exhaustively documented later in Blog entries), to commit the record, it is a simple matter of clicking the tick box in the right hand corner of the list object:

5.PNG

Alternatively, clicking the opposing button will roll back the change:

6.PNG

To update an entry in the List control while browsing back and forth via the navigation buttons, it is a simple matter of clicking the pen icon at the bottom right hand corner of the list entry:

7.PNG

On Clicking the pen Icon, the entry will be overlay by a form entry, presenting the existing values for update:

8.PNG

Upon updating the form, the entry can be updating in the same manner as the addition, by clicking the tick icon:

5.PNG

Alternatively rolling back the change by clicking the opposing button:

6.PNG

To delete an entry, navigate to the entry in the entry in the list control that requires deletion and then click on the trash icon:

9.PNG

A message box will be displayed to confirm that it is the users intention that the entry is to be deleted, upon confirmation, the entry will be deleted:

10.PNG

3) Posting XML and JSON in Postman

This Blog entry is from the Getting Started in Jube section in Learn Jube.

To develop a familiarisation in messaging the Jube platform a number of models are pre-configured in the platform with their messaging formats available in the following document:

http://download.jube.io/formats.pdf

These formats serve to introduce message formats the the different switches that are available to message processing. These formats will also be used regularly in subsequent Blog entries.

In this Blog entry, an Account Financial Transaction will be explored.

Firstly an XML post will be made with the querystring key value pair as follows:

EntityAnalysisModelGUID=253e427e-4523-40d2-b155-fcd5348e20e9

This querystring key value pair would be used in concert with the full endpoint specification if it was called from a programming language or a external HTTP post tool or utility:

https://api.jube.io?EntityAnalysisModelGUID=253e427e-4523-40d2-b155-fcd5348e20e9

To get started with the XML message, navigate to Administration >>> HTTP Post Tools >>> Postman:

1.PNG

In this example, XML is going to be posted. Click the XML radio button to ensure that the XML is formatted in the Postman page properly:

2.PNG

In the Format field, enter the querystring Key Value pair:

EntityAnalysisModelGUID=253e427e-4523-40d2-b155-fcd5348e20e9
3.PNG

Using the formats document, in this example, copy the Sample XML Account Financial Transaction section of the Formats document:

4.PNG

Pasting it into the Request code editor in the Postman page of Jube:

5.PNG

The code editor will parse the XML and give an indication of errors in structure. The message is now ready to be sent to the https://api.jube.io endpoint. Click the POST button:

6.PNG

The Postman page will simply relay the post to the https://api.jube.io endpoint as if it were being called from a programming environment or alternative HTTP post utility. The response will be written out to the response code editor:

7.PNG

The response XML is written out to the Response code editor, formatted as XML. Furthermore the response time is recorded, which is 6 milliseconds in this example, alongside the HTTP response status of 200 (success).

Refresh the page by navigating to Administration >>> HTTP Post Tools >>> Postman then proceed to repeat this process using JSON as follows.

Jube accepts both XML and JSON via the https://api.jube.io endpoint. Sending JSON messages is the same as XML, except the code editor would need to be changed to JSON by clicking on the radio button in the format section of the Postman page:

8.PNG

Although the switch is the same, a different value is used containing instructions on how to parse the JSON. In the formats field enter the querystring key value pair as follows:

EntityAnalysisModelGUID=7e5f96df-5a9b-4e13-a007-e02d79f970b3
9.PNG

This time, use the Sample JSON from the Account Financial Transaction section of the Formats document:

10.PNG

Pasting it into the Request code editor in the Postman page of Jube:

11.PNG

The code editor will parse the JSON and give an indication of errors in structure. The message is now ready to be sent to the https://api.jube.io endpoint. Click the POST button:

12.PNG

The response JSON is written out to the Response code editor, formatted as JSON.

It can be seen that Jube is able to accept both XML and JSON over the https://api.jube.io endpoint. The endpoint is described as being promiscuous in this regard and simply expects the the XML or JSON to be valid, and parse on receipt.

Make note that the structure of both XML and JSON is comprised of elements which can be referenced using XPath or JSON Path respectively.

2) Introducing Postman

This Blog entry is from the Getting Started in Jube section in Learn Jube.

Integration can of course be accomplished by any tool or programming framework that facilitates HTTP interactions with a web server (e.g. .Net, Java, PHP etc). For testing it is quite common to use a HTTP Post Tool, Utility or Client application. Jube has created a page to support manual testing of HTTP messaging in the application, called Postman.

To navigate to Postman, navigate through the the menus Administration >>> HTTP Post Tools >>> Postman:

1.png

The Postman page will be presented:

2.PNG

There are three input parameters worthy of discussion in this Blog entry, keeping in mind that subsequent Blog entries will set about posting to the pre-configured models. The first parameter is represented as radio buttons and select the format of the message to be posted:

3.PNG

The Format option has no bearing on the message being submitted to the server via HTTP and only serves to tell the page how to format the input in the code editor, which affords certain parsing and formatting in the input box.

The second parameter is the URL that the message is to be posted to, which hard codes the Jube https://api.jube.io endpoint and query string delimiter, then expects only querystring parameters separated by the & character:

4.PNG

The final parameter is the Request body, which accepts ether XML or JSON to be posted to the endpoint alongside the querystring parameters:

5.PNG

An important consideration when deciding to use the embedded utility over another tool is that all requests and responses are logged to a database available to Jube support staff. Should any requests not behave as expected, then Jube support can provide more timely intervention having access to the messages and internal error messages.

As mentioned, there are several pre-configured models which exist for the purposes of training and integration testing and comprise the content of several subsequent Blog entries.

1) HTTP Endpoint Request Payload and Switching

This Blog entry is from the Getting Started in Jube section in Learn Jube.

The Jube Platform does not have a hard-coded API specification and the HTTP Endpoint is promiscuous in the format of the GET or POST body that it accepts, with that POST body being parsed as per a specification configured in each Symbol or Entity model.

The POST body may be XML or JSON, in any structure, with its precise structure being configured in the Entity or Symbol model definition of Jube. It of course follows that the XML or JSON should pass a parse on receipt of the POST, appropriate to the specification. When the Jube Engine receives a HTTP request, the POST body is not initially inspected or parsed, instead only the Query String of the HTTP request is inspected for the presence of switching keys.

An example of a HTTP request to the Jube Engine is as follows:

http://api.jube.io:5555?EntityAnalysisModelGUID=253e427e-4523-40d2-b155-fcd5348e20e9

In the above url the following parameters are implied:

Component

Text

Description

SSL Encryption

http:// or https://

The http SSL designator which is used to encrypt HTTP traffic using SSL.

Endpoint

api.jube.io

The IP address or sub domain name for which HTTP posts are to be made.

Port

:443

The colon designating a nonstandard port to be used, followed by the port.

This is not needed, but is included for completeness.

Query String Delineation

?

The model to be invoked for the purposes of processing the POST body is passed in the query string, everything that follows the ? will be considered model parameters.

Model Type Switch

EntityAnalysisModelGUID=

While there are several parameters available in model recall, such as the ability to process a model asynchronously, for the purposes of this document there is a single parameter of EntityAnalysisModelGUID which, by virtue of its presence, switches to the entity subsystem and will seek a GUID as follows.

GUID

253e427e-4523-40d2-b155-fcd5348e20e9

The GUID of the model, created automatically on model insert.

Switching, depending up the presence of a given Switch key, takes place in the following order:

Switch

Description

Symbol

A switch to be made to the Symbol Subsystem to process a new Symbol (e.g. Stock Price).

ExhaustiveSearchInstanceGUID

A switch to be made to the Exhaustive Subsystem to recall a trained model, bypassing the Entity Model and Symbol Model processing.

EntityAnalysisModelGUID

A switch to be made to the Entity Subsystem to process a new Entity (e.g. Account Transaction).

EntityAnalysisModelGUID and Tag

A switch to be made to process a Tag for a given transaction previously processed through an Entity Model.

Although the endpoint for all requests made to Jube is the same, the processing is very different depending upon the switch.

Jube has several pre-configured models for the purposes of validation, integration testing and confidence building. These pre-configured models will be explored in subsequent Blog entries. The pre-configured models and example switches and formats are contained in the formats document as follows:

http://download.jube.io/formats.pdf

The formats document provides examples of messages routed by each switch set out above.