Feedback Loops & Label API: Labeling and feedback best practices

At its heart, machine learning algorithms learn from good and bad examples. Our machine learning solutions learn from transaction labels and take states resulting from rules or manual investigations into account. Using positive and negative transaction labels to give the most accurate feedback and rule out false positives. Providing feedback labels to SEON Machine Learning to optimize the performance and customize to the use case you want to apply Blackbox Score and Machine Learning Rules

SEON has two ways to provide feedback to machine learning: labels and transaction states. Ideally, SEOM ML should be trained on labeled, verified transactions to have a clear view of good and bad events. However, both Blackbox and Whitebox ML are configured by default to learn both from labels and transaction states. This helps you to train models quickly while you start the comprehensive labeling.

States

Transaction states indicate where the transaction is safe, needs attention, or must be blocked. States also indicate the action taken by the SEON system with the transaction, considering states like traffic lights.

States are assigned to transactions immediately in real-time by SEON’s decision engine, the scoring engine when evaluating rules. Transaction states can be changed manually.

Labels

Transaction labels are the ultimate verification of a transaction's truth. Consider them as the final positive or negative outcome for the transaction. Transactions are labeled after you have verified them good or bad via your business, your systems and your processes outside SEON. 
Labels containing the final truth are usually added to SEON with a delay that your business defines: how much time is required for the clear verification. If you are running manual reviews, the outcome of the fraud analyst review will also be a verification label, depending on the time spent on this work. If you are a lending or BNPL business, the first installment due to pay typically defines this delay.
Labeling both negative and positive signals is important: this will provide the most training data about your business's risk profile. Since malicious events are much rarer than legitimate ones, negative labels (like fraud, chargeback, and default) are naturally less frequent than positives. Our algorithms take this distribution into account.

Label API

A transaction can have a single label (after Label API v2 version) to ensure that it is clearly assigned to the verified reason for setting its state. This single label can be either a positive or a negative one indicating the most precise known reason for making it so using the set of label values offered by SEON. If you would like to apply additional categorization or information beyond the label outcome, leverage Tags to apply multiple categories to a Transaction. More information on SEON’s Tag API can be found here: https://docs.seon.io/api-reference/tag-api  
To ensure the most complete coverage of labeling, we recommend using our Label API to feed transaction labels to SEON at scale. 
SEON’s Label API v2 is optimized to fully harness SEON’s powerful Machine Learning to make rule suggestions and ML scoring completely adapted to your business. This is why it has a broad set of label values available to use. Labels are grouped into use cases such as general fraud detection, E-commerce, and Credit Risk (incl. BNPL), with more to come. Only the usage of these labels is supported across SEON, however, if your use case requires a label not listed please contact us to add it or find the best match.
See our full Label API v2 documentation here: https://docs.seon.io/api-reference/label-api-v2 

Manual Labeling

You can also manually add labels at the Admin panel’s transaction list page. Here, you have to use the state changer dropdown to add the label as a reason for the transaction's state.

When setting up labels for your use case, you have to enable the labels you want to see and be applicable on the Admin Panel. Please visit Settings/Machine Learning Settings, where the Label Settings section contains the label set with toggles.
 

Four ways of efficient feedback using labels

(1) Continuous labeling

• Add labels to transactions on an ongoing basis if you want SEON ML to adapt to your business's changes and always catch recent fraud patterns.
  • Good practice: Automatically label each transaction as soon as verification is completed on your side using the Label API or the Admin Panel. It is also acceptable to load frequent batches of labels through the API daily or weekly.
  • Bad practice: add one time, quarterly or less frequent batches of labels.

(2) Right-in-time labeling

• To reduce the time required to adapt to new patterns, provide the labels right away when your business has the verification. The longer the delay, the slower the ML will adapt to recent and, thus, even ongoing fraudulent activities. 
  Consider that blackbox machine learning retrains on a daily basis while whitebox can even get retrained multiple times a day. Adding verification already available on your side with more delay than these cadences introduces delay and, thus, less accuracy in capturing fraud.
  • Good practice: add verification labels as soon as they are made through the label API
  • Bad practice: collect verifications into infrequent batches (monthly, quarterly or more)

(3) Precise labeling

• The Label API v2 introduces a label set that enables you to mark the reasons of verification precisely. Using the most precise labels for verifying the reason of the transaction state by the given type of fraud or malicious activity enables SEON to build specifically sensitive models for these in the future. This is a joint investment in ML innovation for more powerful fraud detection.
  • Good practice: adding labels encoding the reason for decision within the use case of the business (e.g. fraud detection: bonus abuse)
  • Bad practice: using single generic labels for all types of negative decisions made. (e.g.  fraud detection: fraud in all cases)

(4) Balanced labeling

• As said, machine learning is trained on both good and bad examples. For optimal model performance, both positive and negative labels must be added. Naturally, there should be many more good transactions than bad ones.
  • Good practice: All transactions have a label added, both positive and negative decisions marked (it is also acceptable to have the majority of positives labeled while all negative ones are marked).
  • Bad practice: add negative labels only to mark rejected transactions.

Timeline overview of the good labeling practice enabling machine learning

When starting to use SEON, you usually have default rules, some custom rules, and the Blackbox score calculated by the base model.
Using SEON for decision-making (approving or declining transactions) will start providing the first information to our ML models about what is good or bad in your business. To have the training for these specificities, first the blackbox and the whitebox ML are both trained on states and labels.
Labeling must be implemented to add the true, verified reasons for decisions or outcomes of manual reviews. Feed labels in automatically using the Label API and add outcomes from manual reviews using the Admin Panel. 
After having labeling in place for the majority of the transactions, it is reasonable to optimize machine learning by training the models only on labeled transactions. The setting is available for blackbox and whitebox ML in Settings/Machine Learning Settings on the Admin Panel.
Marking transactions for purposes other than training ML
Labels are used to provide verified good and bad transactions from which SEON ML can learn. However, there are analytics, reporting, or various other reasons that require transactions to be marked and grouped together. Tagging is a flexible marking feature through the API and the Admin panel that assigns any free text tags to transactions and searches or filters for these. You can even use tags in rules.

Tags can be used in combination with labels:

• Mark transactions to train precise ML scores and rules using labels. The label contains the reason for the decisions made and is used in training.
• Mark transactions for reporting or else using tags – Any number of tags per transaction contain information you add to the transaction for further filtering or rules.