NSTA Explorer -- Phase 2 methodology

How gradient‑boosted trees learn

The mechanism behind Phase 2's XGBoost classifier and a Sw regressor built the same way: not one tree guessing the answer, but a relay of shallow trees, each one built purely to correct what the trees before it got wrong. Worked through with real numbers on five depth samples, predicting water saturation (Sw) from resistivity and porosity.

The five depth samples

A held-out well isn't used for training — but for this walkthrough, imagine five depth samples with a true Sw value each (say, from a nearby DST-confirmed interval). Every tree round starts from these same rows.

Depth (m)Rt (Ω·m)φ (porosity)True Sw
2041.5840.190.18
2043.0220.210.46
2044.560.240.81
2046.030.220.94
2047.5610.200.24

Three boosting rounds

Round 0 is a flat guess — just the average Sw across all five rows, before any tree has looked at a single curve. Each round after that: measure the residual (true − current prediction), grow one small tree whose entire job is to predict that residual, then add a shrunk version of its output onto the running total.

0starting guess

Every row predicted as the mean

a
0.526
b
0.526
c
0.526
d
0.526
e
0.526
No curves used yet — just mean(0.18, 0.46, 0.81, 0.94, 0.24) = 0.526. Every row is equally wrong.
1tree 1

Residual to fix

a
−.346
b
−.066
c
+.284
d
+.414
e
−.286
→

asks: is Rt < 15?

A shallow tree splits the five rows on Rt and φ, trying to predict the residual above — not the true Sw itself.
→

Running prediction (+ shrunk tree output)

a
0.35
b
0.35
c
0.73
d
0.73
e
0.35
2tree 2

Residual still left

a
−.170
b
+.110
c
+.080
d
+.210
e
−.110
→

asks: is φ > 0.205?

Tree 2 has never seen Tree 1's split — it only sees what's still wrong after Tree 1's correction.
→

Running prediction

a
0.22
b
0.44
c
0.78
d
0.90
e
0.26

Where it lands after 200 rounds

Two rounds already pulled the prediction from a flat 0.526 to within a few hundredths of the true Sw. Phase 2 doesn't stop at 2 rounds — it runs 200, each one nudging the total a little further, with a shrinkage factor (the learning rate) keeping any single tree from overcorrecting.

1.0 0.5 0 boosting round → true 0.94 (d) true 0.18 (a)
predicted Sw, sample d (true 0.94) predicted Sw, sample a (true 0.18) round-0 starting guess (0.526, both rows)
Why not one big tree, or a Random Forest? One deep tree memorizes the five training rows perfectly and generalizes badly to a new well. A Random Forest builds many trees too — but independently, each on its own random slice of data, then just averages their votes at the end; no tree ever sees another tree's mistakes. Boosting is sequential on purpose: every tree's entire reason for existing is the error left behind by all the trees before it, which is why order matters here and doesn't in a forest.