Feedback Loops & Label API: Labeling and feedback best practices

At its heart, machine learning algorithms learn from good and bad examples. Our machine learning solutions’ performance can be boosted significantly by providing transaction labels based on outcomes of manual investigations or downstream evaluations by systems such as payment processors or credit checks . Using positive and negative transaction labels gives the most accurate feedback and rules out false positives. Providing feedback labels to SEON Machine Learning to optimize the performance and customize to the use case you want to apply the Blackbox Score and Machine Learning Rules.

SEON has two ways to provide feedback to machine learning: labels and transaction states. Ideally, SEOM ML should be trained on labeled, verified transactions to have a clear view of good and bad events. However, both Blackbox and Whitebox ML are configured by default to learn both from labels and transaction states. This helps you to train models quickly while you start the comprehensive labeling.

States

Transaction states indicate where the transaction is safe, needs attention, or must be blocked. States also indicate the action taken by the SEON system with the transaction, considering states like traffic lights.

States are assigned to transactions immediately in real-time by SEON’s decision engine, the scoring engine when evaluating rules. Transaction states can be changed manually or using automation via the Label API when adding a label with a different signal (negative instead a positive).

Labels

Transaction labels are the ultimate verification of a transaction's truth. Consider them as the final positive or negative outcome for the transaction. Transactions can be labeled after you have verified them good or bad via your business, your systems and your processes outside SEON. 
Labels containing the final truth are usually added to SEON with a delay that your business defines: how much time is required for the clear verification. If you are running manual reviews, the outcome of the fraud analyst review will also be a verification label, depending on the time spent on this work. If you are a lending or BNPL business, the first instalment due to pay typically defines this delay.
Labeling both negative and positive signals is important: this will provide the best training data about your business's risk profile. Since malicious events are much rarer than legitimate ones, negative labels (like fraud, chargeback, and default) are naturally less frequent than positives. Our algorithms take this distribution into account.

Label API

A transaction can have a single label to ensure that it is clearly assigned to the verified reason for setting its state. This single label can be either a positive or a negative one, indicating the most precise known reason for making it so using the set of label values offered by SEON. If you would like to apply additional categorization or information beyond the label outcome, leverage Tags to apply multiple categories to a Transaction. More information on SEON’s Tag API can be found here: https://docs.seon.io/api-reference/tag-api .
To ensure the most complete coverage of labeling, we recommend using our Label API to feed transaction labels to SEON at scale. Labels can overwrite or confirm the original State SEON provided when the transaction took place. Using the Label API is free of charge.
SEON’s Label API v2 is optimized to gain access to the best accuracy of SEON’s powerful Machine Learning to make rule suggestions and ML scoring completely adapted to your business. This is why it has a broad set of label values available to use. Labels are grouped into use cases such as general fraud detection, E-commerce, and Credit Risk (incl. BNPL), with more to come. Only the usage of these labels is supported across SEON, however, if your use case requires a label not listed please contact us to add it or find the best match.
See our full Label API v2 documentation here: https://docs.seon.io/api-reference/label-api#label-api-request .

Manual Labeling

You can also manually add labels in the Admin panel’s transaction list page. Here, you have to use the state changer dropdown to add the label as a reason for the transaction's state.

When setting up labels for your use case, you have to enable the labels you want to see and be applicable on the Admin Panel. Please visit Settings/Machine Learning Settings, where the Label Settings section contains the label set with toggles.
 

Tip: Labeling on the Admin UI and using the Label API are completely cost-free and highly advised to be practiced, as it is our mutual aim to provide the best solution to you.

Four ways of efficient feedback using labels

To unlock the best machine learning performance, we recommend the following ways to use labels in SEON. This ensures that both the blackbox and the whitebox machine learning algorithms utilize the most accurate and up-to-date knowledge shared to learn from and provide the best-performing and bespoke scores and ML rules.

(1) Continuous labeling

• Add labels to transactions on an ongoing basis if you want SEON ML to adapt to your business's changes and always catch recent fraud patterns.
  • Good practice: Automatically label each transaction as soon as verification is completed on your side using the Label API or the Admin Panel. It is also acceptable to load frequent batches of labels through the API daily or weekly. Both ways are completely cost-free and highly advised to be practiced, as it is our mutual aim to provide the best solution to you.
  • Bad practice: add one time, quarterly or less frequent batches of labels. By doing so, the system would have a hard time keeping up with the new and changing trends of fraudulent activities.

(2) Right-in-time labeling

• To reduce the time required to adapt to new patterns, provide the labels right away when your business has the verification. The longer the delay, the slower the ML will adapt to recent and, thus, even ongoing fraudulent activities. 
  Consider that blackbox machine learning retrains on a daily basis while whitebox can even get retrained multiple times a day. Adding verification already available on your side with more delay than these cadences introduces delay and, thus, less accuracy in capturing fraud.
  • Good practice: add verification labels as soon as they are made through the label API
  • Bad practice: collect verifications into infrequent batches (monthly, quarterly or more)

(3) Precise labeling

• The Label API v2 introduces a label set that enables you to mark the reasons of verification precisely. Using the most precise labels for verifying the reason of the transaction state by the given type of fraud or malicious activity enables SEON to build specifically sensitive models for these in the future. This is a joint investment in ML innovation for more powerful fraud detection.
  • Good practice: adding labels encoding the reason for decision within the use case of the business (e.g. fraud detection: bonus abuse)
  • Bad practice: using single generic labels for all types of negative decisions made. (e.g.  fraud detection: fraud in all cases)

(4) Balanced labeling

• As said, machine learning is trained on both good and bad examples. For optimal model performance, both positive and negative labels must be added. Naturally, there should be many more good transactions than bad ones.
  • Good practice: All transactions have a label added, and both positive and negative decisions are marked (it is also acceptable to have the majority of positives labeled while all negative ones are marked).
  • Bad practice: add negative labels only to mark rejected transactions.

Timeline overview of the good labeling practice enabling machine learning

When starting to use SEON, you usually have default rules, some custom rules, and the Blackbox score calculated by the base model.
Using SEON for decision-making (approving or declining transactions) will start providing the first information to our ML models about what is good or bad in your business. To have the training for these specificities, first the blackbox and the whitebox ML are both trained on states and labels.
Labeling must be implemented to add the true, verified reasons for decisions or outcomes of manual reviews. Feed labels in automatically using the Label API and add outcomes from manual reviews using the Admin Panel. 
After labeling in place for the majority of the transactions, it is reasonable to optimize machine learning by training the models only on labeled transactions. The setting is available for blackbox and whitebox ML in Settings/Machine Learning Settings on the Admin Panel. Add labels on final decision outcomes different from the SEON States set by the rules would ensure that both of our machine learning models can focus on cases when an incorrect decision has been provided by the ruleset and will aim to fill in the gaps in your personalized decision-making model.
Marking transactions for purposes other than training ML
Labels are used to provide verified good and bad transactions from which SEON ML can learn. However, there are analytics, reporting, or various other reasons that require transactions to be marked and grouped together. Tagging is a flexible marking feature through the API and the Admin panel that assigns any free text tags to transactions and searches or filters for these. You can even use tags in rules.

Tags can be used in combination with labels:

• Mark transactions to train precise ML scores and rules using labels. The label contains the reason for the decisions made and is used in training.
• Mark transactions for reporting or else using tags – Any number of tags per transaction contain information you add to the transaction for further filtering or rules.