Structuring & Evaluating Trading Data

Most traders collect data. They log their trades, track their wins and losses, and calculate percentages. From this information, they assume they can determine whether or not they have an edge.

The problem is that data alone does not prove predictive value. If changing market conditions or changing scenarios does not impact outcome distribution, then the strategy does not possess a structural edge — it is simply producing variance.

An edge is not defined by a high win rate. An edge is defined by a difference in probability under different conditions. Understanding this distinction is critical.

Many traders attempt to jump directly into measuring profitability or expected value. However, doing so without first understanding probability can lead to false conclusions.

Profitability answers a simple question: did this make money? Probability answers a different and far more important question: does behaviour change under different conditions? Without identifying whether behaviour changes across different environments, it becomes extremely difficult to determine whether profitability is the result of true structural advantage or merely random variance.

Probability reveals whether behaviour changes. Profitability reveals whether that behavioural change can be monetised. You must first understand behaviour before you can confidently monetise it.

Within the broader quant-inspired strategy pipeline, we are currently in the validation stage — determining whether the original hypothesis holds significance. If you have been following the process so far, you should have: selected an anchor event, defined the features of the strategy, established market regimes, constructed a binary hypothesis, defined variables, and collected impartial historical data.

The purpose of this episode is to transform that raw data into structured probability.

At this stage, your collected data should resemble a structured log. Each example should include whether the outcome was valid or invalid, the date of the scenario, screenshots of the relevant charts, and the variables present during the event. Screenshots and dates are primarily for traceability — they allow you to revisit and verify the exact market scenario if necessary.

For now, the only important components are valid outcomes, invalid outcomes, and the variables associated with each example.

To convert raw observations into probabilities, the data must first be grouped according to how the hypothesis is structured. In this model, the hypothesis is defined by three core components: initiation liquidity, supplementary confluences, and market regime.

Rather than measuring every possible combination immediately, the first step is to measure each variable independently. Group all examples where a specific initiation liquidity occurs. Count how many resulted in valid outcomes and how many resulted in invalid outcomes. Calculate the probability of a valid outcome. Repeat this for every variable individually. Once completed, each variable will have its own probability distribution.

A practical way to organise this information is by creating separate pages for each variable — a page for 15-minute swing liquidity as initiation, a page for fair value gap confluence, a page for specific market regimes, and so on. Within each page, valid examples appear at the top, invalid examples below, with totals recorded and probabilities calculated.

For example: 494 valid outcomes and 207 invalid outcomes produces a 71% probability of a valid outcome under that variable. Once completed across all variables, you will have a clear probability distribution for every structural component of the strategy.

While analysing variables individually provides useful insights, it does not tell the complete story. Variables do not operate independently — their behaviour depends on the interaction between conditions.

Consider the question: what makes a car fast? Suppose three contributing factors are engine size, tyre type, and road surface. If we measure engine size alone, we might conclude that larger engines produce faster cars. But a large engine combined with bald tyres on an icy road will not produce a fast car. By measuring variables independently, we produce an averaged result that does not accurately represent individual scenarios. This is exactly what happens when trading variables are measured in isolation.

The next step is to measure variable combinations. Instead of grouping by individual variables, we group by overall scenarios. Select a specific initiation variable, subdivide by confluence type within that group, observe market regimes within each of those groups, and measure valid and invalid outcomes for each combination. At the end of this process, every scenario will have its own probability.

Once probabilities exist for each scenario, you can determine whether variables actually influence outcomes. Hold all variables constant except one, change the variable being tested, and compare the resulting probability distributions. If altering the variable produces a meaningful difference, the variable likely holds predictive value. If probabilities remain similar, the variable likely does not influence the outcome.

If analysis reveals that certain variables behave nearly identically, they should be combined. Combining variables increases sample size, reduces dataset fragmentation, and simplifies the strategy. This is one of the major benefits of conducting probability analysis before building a final trading model.

A meaningful edge does not need to produce dramatic results. Most professional trading edges are small but consistent. Not every scenario needs to be traded — the objective is to identify the scenarios where probability provides a meaningful advantage.

This process may appear time-consuming, but it serves a critical purpose. Probability analysis isolates behavioural causality. Without isolating behavioural causes first, it is impossible to determine whether profits come from structural edge, trading skill, or random variance. Long-term traders must know that their profits come from structure, not luck.

During the process of collecting and analysing data, you will inevitably discover new observations about your anchor event. Many traders make the mistake of rebuilding their entire strategy around these observations. Instead, the correct approach is to test each observation individually. Episode 08 introduces the concept of sub-hypotheses — engineering and testing new behavioural observations without corrupting the original dataset.