Target Audit
The target variable is audited before modeling to establish what is actually being measured.
A research-grade investigation into whether leakage-controlled search-demand, competition, freshness, content-production, and contextual signals contain generalizable information about observed content momentum.
This study investigates whether a leakage-controlled set of content, search-demand, competition, freshness, and content-production signals can explain observed content momentum and support a transparent prioritization framework for human content review.
The analysis deliberately separates predictive modeling from exploratory opportunity ranking. Identifiers are excluded from the feature matrix, target-derived variables are removed, missing-value treatment is learned inside pipelines, and grouped train/test validation is used as the primary estimate because observations belong to anonymized clients.
The resulting evidence does not support a claim that the tested machine-learning models provide generalizable predictive improvement over the mean baseline under grouped held-out validation. The operational contribution is therefore framed as an auditable decision-support layer rather than an autonomous forecasting system.
The capstone is intentionally structured around validity, reproducibility, interpretability, and disciplined inference.
The target variable is audited before modeling to establish what is actually being measured.
Variables that directly encode or temporally entangle the target are excluded.
Client groups are separated between training and testing to prevent optimistic leakage across related observations.
A mean dummy regressor is mandatory, preventing weak models from being presented as useful merely because they are technically non-trivial.
Missing-value treatment and preprocessing are learned inside modeling pipelines after the split.
Final predictive metrics are evaluated on held-out data rather than training observations.
Predictive modeling and opportunity ranking are explicitly treated as different analytical tasks.
The analysis does not claim causality, Google-algorithm knowledge, or guaranteed future performance.
Under grouped held-out validation, no tested machine-learning model improved upon the mean baseline. Ridge Regression was the strongest non-dummy model by MAE, but its MAE remained above the baseline. The appropriate interpretation is therefore limited predictive signal under the tested design—not failure of the research.
This distinction is fundamental: a rigorous empirical study should preserve evidence even when the evidence does not produce an impressive predictive score.
The notebook reconstructs trend_pct from the relationship between impressions in the last 30 days and the previous 30 days.
26,612 rows were tested. The median absolute difference was approximately 0.022 percentage points, with 100% of audited rows matching within 0.1 percentage point.
The workflow moves sequentially from data audit to target validation, feature construction, grouped modeling, diagnostic analysis, and finally a transparent opportunity index.
Load the anonymized content warehouse extract and establish its schema, missingness, identifiers, and dimensional structure.
Determine whether available fields mathematically or temporally encode the target and remove them from predictive modeling.
Train Dummy, Ridge, Random Forest, and Extra Trees models under client-grouped held-out validation.
Construct a transparent opportunity score using demand, staleness, competition, and lifecycle signals.
The mean dummy regressor remains the strongest model by MAE. Ridge Regression is the best non-dummy model, but its error is still higher than the baseline.
| Model | MAE | RMSE | R² | Interpretation |
|---|---|---|---|---|
| Dummy Regressor | 65.4752 | 295.9562 | -0.0005 | Baseline |
| Ridge Regression | 71.3596 | 297.5412 | -0.0113 | Best non-dummy model |
| Random Forest | 88.6801 | 309.0226 | -0.0908 | Limited predictive signal |
| Extra Trees | 88.9691 | 309.3702 | -0.0933 | Limited predictive signal |
Because the best non-dummy model has higher MAE than the baseline, the machine-learning system should not be presented as a validated predictive engine for future content momentum.
The section below reads the embedded PNG outputs directly from capstone.ipynb. Keep the notebook in the same directory as this HTML file so the original analytical figures can be rendered without redrawing or approximating them.
These values are diagnostic rather than causal explanations. They describe the change in model scoring loss under permutation within the exploratory analysis.
| Feature | Importance Mean | Importance Std |
|---|---|---|
| word_count | 10.925458 | 0.520312 |
| search_volume | 10.855314 | 0.973300 |
| char_count | 5.447605 | 0.413041 |
| competition | 3.428014 | 0.381740 |
| days_since_last_update | 2.959371 | 0.262951 |
| cpc | 2.074102 | 0.207187 |
| content_age_days | 1.845638 | 0.945165 |
| provider_used | 0.022146 | 0.001481 |
| competition_level | -0.088470 | 0.027482 |
| main_intent | -0.440405 | 0.189133 |
| content_type | -0.531634 | 0.037549 |
| model_used | -4.921439 | 0.301586 |
The opportunity engine is explicitly not presented as a forecast. It is an interpretable review-priority index designed to help human analysts decide which content deserves attention first.
Percentile-ranked search volume. Higher demand increases review priority.
Percentile-ranked days since the content was last updated.
Percentile-ranked competition signal from the available dataset.
Percentile-ranked content age as an additional review signal.
The ranking system sorts eligible observations by opportunity score and assigns a human-review priority.
The notebook identified 27,532 rows eligible for opportunity scoring.
Reason codes such as "Stale Content", "High Search Demand", and "Mature Content" are deliberately diagnostic.
They do not imply causality or guarantee future traffic, ranking, click, or business outcomes.
These comparisons are descriptive rather than causal. They characterize how the opportunity index behaves across groups.
| Content Type | Pages | Mean Score | Median Score | Mean Staleness | Mean Demand |
|---|---|---|---|---|---|
| keyword article | 26,835 | 50.6286 | 49.2899 | 47.8334 | 163.0091 |
| comparison article | 697 | 25.8707 | 27.1389 | 18.0100 | 0.0000 |
| Intent | Pages | Mean Score | Median Score | Mean Staleness | Mean Demand |
|---|---|---|---|---|---|
| Transactional | 5,691 | 59.9529 | 59.6194 | 52.9181 | 90.6783 |
| Commercial | 4,561 | 58.5345 | 58.4936 | 50.6895 | 166.9458 |
| Navigational | 46 | 54.0099 | 57.8683 | 41.0217 | 63.0435 |
| Informational | 17,092 | 44.5520 | 43.5909 | 44.2206 | 181.0180 |
A secondary random row split was performed as a sensitivity analysis. Grouped validation remains the primary estimate because observations are nested within anonymized clients.
| Validation Design | MAE | RMSE | R² |
|---|---|---|---|
| Grouped by client | 88.9691 | 309.3702 | -0.0933 |
| Random row split | 64.7979 | 317.9670 | -0.0128 |
The difference demonstrates why grouped validation is retained as the primary design rather than selecting whichever split produces the most flattering metric.
"The resulting system should be presented as a research-backed decision support layer for content review, not as an autonomous optimizer or a validated forecast of search performance."
The capstone establishes two principal contributions.
First: it tests whether historical content, search, competition, freshness, and production attributes contain generalizable predictive information about observed content momentum.
Second: it converts available search-intelligence context into a transparent, auditable opportunity-review system without overstating predictive or causal capability.
The study therefore prioritizes methodological defensibility over artificially impressive model performance.
The notebook uses a fixed random seed of 42 and exports public-safe analytical artifacts for inspection and repository versioning.
The website is the presentation layer. The notebook and repository remain the primary reproducibility artifacts.