UK-based online statistics and data analysis support for USA, UK, and international clients. No exams, no impersonation, no fabricated data.
Regression Tests and Models

Generalized Least Squares Regression: Interpretation, SPSS, Python, R and Excel Guide

Heteroscedasticity, Error Covariance, Inverse-Variance Weighting and Feasible GLS Generalized Least Squares Regression: Interpretation, SPSS, Python, R and Excel Guide Generalized Least Squares Regression extends ordinary least...

Statistics guide Ethical learning support SPSS/R/Python/Excel friendly
Generalized Least Squares Regression: Interpretation, SPSS, Python, R and Excel Guide

Heteroscedasticity, Error Covariance, Inverse-Variance Weighting and Feasible GLS

Generalized Least Squares Regression: Interpretation, SPSS, Python, R and Excel Guide

Generalized Least Squares Regression extends ordinary least squares when regression errors do not have equal independent variance. This guide compares OLS, feasible GLS/WLS, diagonal-covariance GLS and GLSAR using G3 final grade as the outcome. It explains the covariance matrix, variance-model weights, Breusch–Pagan and White tests, Durbin–Watson, coefficients, nine Python charts, nine R validation discussions, the current SPSS export problem and a separate formula-driven Excel example.

AdvertisementGoogle AdSense top placement reserved here

Quick Answer: Generalized Least Squares Regression Result

The verified Python and R analyses use 649 complete observations. The outcome is G3. Numeric predictors are G1, G2, studytime, failures, absences, age and Medu. Categorical predictors are school, sex and address. The baseline OLS residuals show strong heteroscedasticity, so inverse estimated error-variance weights are used for feasible GLS.

In the Python report, OLS has R² = 0.8524 and adjusted R² = 0.8501. Feasible GLS/WLS has R² = 0.8695 and adjusted R² = 0.8674. Its AIC decreases from 2143.3400 to 2037.0053, and BIC decreases from 2192.5698 to 2086.2351. MAE improves from 0.7791 to 0.7721, while unweighted RMSE changes slightly from 1.2403 to 1.2423.

Rows used649
Model slope df10
OLS adjusted R²0.8501
FGLS adjusted R²0.8674

Breusch–Pagan p0.000157
White p0.000348
Durbin–Watson1.8723
GLSAR ρ0.0661

OLS MAE0.7791
FGLS MAE0.7721
OLS RMSE1.2403
FGLS RMSE1.2423

Main conclusion: Breusch–Pagan and White diagnostics reject constant OLS error variance. Feasible GLS produces a substantially larger weighted R² and lower within-model AIC/BIC, while modestly improving MAE. The correction is motivated by heteroscedasticity, not strong serial correlation, because Durbin–Watson is close to 2 and the GLSAR AR(1) estimate is only 0.0661.

Important fit-metric caution: weighted and unweighted R², likelihood values and residual-error summaries are not always directly comparable across OLS and WLS/GLS. The FGLS weighted R² improvement is meaningful for the weighted objective, but the slight increase in unweighted RMSE shows that GLS does not uniformly improve every prediction metric.

SPSS status: the supplied SPSS PDF is not a completed GLS analysis. It contains only the warning “EMBEDBOOKMARKS is not a recognized subcommand” and says execution stopped. The SPSS output must be regenerated with the compatibility-safe PDF export command before coefficient tables can be reported.

Table of Contents

  1. What Is Generalized Least Squares Regression?
  2. Why OLS Can Be Inefficient
  3. GLS Matrix Formula
  4. Feasible GLS and Variance Modelling
  5. GLS vs WLS vs FGLS vs GLSAR
  6. When to Use Generalized Least Squares
  7. Dataset and Variables Used
  8. OLS Diagnostic Results
  9. OLS, FGLS, Diagonal GLS and GLSAR Comparison
  10. Coefficient Interpretation
  11. How GLS Weights Work
  12. GLS Assumptions
  13. Python Chart-by-Chart Interpretation
  14. R Charts, Tables and Independent Validation
  15. SPSS Output Status and Corrected Workflow
  16. Excel Worked GLS Results
  17. Python, R, SPSS and Excel Workflows
  18. Generalized Least Squares Code
  19. GLS vs Robust Standard Errors
  20. Prediction, Fit and Model Selection
  21. APA-Style Reporting
  22. Reporting Checklist
  23. Common Mistakes
  24. Troubleshooting Guide
  25. Downloads and Resources
  26. Related Salar Cafe Guides
  27. FAQs

What Is Generalized Least Squares Regression?

Generalized Least Squares Regression is a linear regression estimator designed for errors with a non-spherical covariance structure. Ordinary least squares assumes that, conditional on the predictors, errors have equal variance and are uncorrelated. GLS allows error variances to differ and errors to be correlated according to a specified covariance matrix.

When the covariance matrix is known, GLS transforms the data so that the transformed errors satisfy the ordinary least-squares conditions. When the covariance structure is unknown and estimated from data, the method is called feasible generalized least squares.

Simple definition: GLS gives observations and residual patterns different influence according to the estimated error covariance, rather than treating every observation as equally precise.

What Changes from OLS?

GLS can change coefficient estimates, standard errors, confidence intervals, p-values and fitted values. It does not merely replace conventional standard errors. If the weighting or covariance model is correct, GLS can be more efficient than OLS. If it is misspecified, the apparent precision gain can be misleading.

Why Ordinary Least Squares Can Be Inefficient

Under heteroscedasticity, OLS coefficient estimates can remain unbiased when the conditional-mean model is correct, but ordinary OLS standard errors become unreliable and the estimator is no longer the most efficient linear unbiased estimator. High-variance observations receive the same influence as low-variance observations even though their outcomes are less precisely measured.

Under serial or clustered correlation, OLS also ignores dependence between errors. The effective information is smaller than the raw record count suggests. GLS addresses these problems by modelling a covariance matrix rather than assuming the identity matrix multiplied by one variance.

Error ProblemOLS ConsequenceGLS ResponseExample
HeteroscedasticityIncorrect conventional SEs and inefficient coefficients.Use observation-specific inverse-variance weights.Error spread increases with fitted G3.
AR(1) serial correlationResiduals depend on the preceding observation.Model off-diagonal covariance using ρ.Time-series residuals persist over periods.
Cluster correlationWithin-cluster observations are not independent.Specify block covariance or use mixed/panel methods.Repeated students within schools.
Known measurement precisionAll observations are treated equally.Weight by known inverse measurement variance.Study estimates with different standard errors.

Generalized Least Squares Matrix Formula

The linear model is:

y = Xβ + ε,    E(ε | X) = 0,    Var(ε | X) = Ω

When Ω is known and positive definite, the GLS coefficient estimator is:

β̂GLS = (X′Ω⁻¹X)⁻¹X′Ω⁻¹y

If Ω = σ²I, the formula reduces to OLS. If Ω is diagonal with observation-specific variances, GLS becomes weighted least squares. If Ω contains off-diagonal correlations, GLS also accounts for error dependence.

Whitening Transformation

Suppose a matrix P satisfies P′P = Ω⁻¹. Multiplying the model by P produces transformed errors with identity covariance:

Py = PXβ + Pε

OLS applied to the transformed outcome and predictors produces the GLS estimator. This interpretation helps explain why GLS is still a least-squares method: it minimizes squared residuals in transformed space.

Feasible Generalized Least Squares and the Variance Model

The complete covariance matrix is rarely known. Feasible GLS first fits OLS, uses OLS residuals to estimate the error-variance pattern, constructs an estimated covariance matrix and then refits the regression using that estimate.

Worked Analysis Variance Model

log(êᵢ² + small constant) = γ₀ + γ₁|ŷᵢ| + uᵢ

The predicted log variance is exponentiated to obtain a positive estimated variance. The feasible GLS weight is:

wᵢ = 1 / σ̂ᵢ²

Observations with smaller predicted error variance receive larger weights. Observations with larger predicted variance receive smaller weights. The full-data weight distribution spans approximately 1.9 to 5.1 in the supplied histogram, with the greatest concentration around 2.7 to 3.4.

Why Log Residual Squared?

Squared residuals are nonnegative and highly skewed. Modelling their logarithm ensures that exponentiated predictions are positive and can stabilize the variance relationship. A small constant prevents taking the logarithm of zero.

One-Step vs Iterated FGLS

The supplied workflow is a practical one-step FGLS correction: OLS residuals estimate variance, and the weighted model is fitted once. Iterated FGLS repeats variance estimation and weighting until convergence. Iteration may improve internal consistency but can also amplify a misspecified variance model.

GLS vs WLS vs FGLS vs GLSAR

MethodCovariance StructureWeights / ParametersUse in This Analysis
OLSσ²IAll observations have equal weight.Baseline model and source of diagnostic residuals.
WLSDiagonal ΩKnown or supplied weights.Computational form used for feasible inverse-variance weights.
FGLSEstimated ΩWeights estimated from OLS residual variance.Main model.
Diagonal GLSEstimated diagonal covariance matrixσ̂ᵢ² on the diagonal.Numerically validates the FGLS/WLS fit.
GLSARAR(p) covarianceρ estimated iteratively.Autocorrelation comparison; ρ = 0.0661.

The feasible WLS and diagonal GLS models produce nearly identical results: Python R² = 0.8695, adjusted R² = 0.8674 and MAE = 0.7721 for both. Their AIC values differ only from 2037.0053 to 2037.0021 because the implementations represent the same diagonal covariance correction.

When to Use Generalized Least Squares Regression

SituationUse GLS?Reason
OLS residual variance changes systematicallyPossiblyFGLS can model the variance and improve efficiency.
Known observation variancesYesInverse known variance is a principled weight.
Time-series errors follow AR(1)Use GLSAR or time-series GLSOff-diagonal covariance matters.
Only standard errors are heteroscedastic but coefficients are the main targetRobust OLS may be preferableIt avoids specifying a variance model.
Weights are arbitrary importance scoresNoGLS weights should represent error precision, not subjective importance.
Outcome is nonnormal or bounded and mean-variance relation is intrinsicConsider GLMA generalized linear model may match the outcome distribution better.
Repeated observations within clustersConsider GEE/mixed modelsCorrelation structure and subject-level interpretation may require another framework.

The decision should begin with the error structure and research goal, not with a desire to obtain a larger R² or smaller p-value.

Dataset and Variables Used

The Python and R reports use the same 649 complete observations and the same formula. G3 is the continuous outcome. Seven numeric predictors and three categorical predictors enter both OLS and FGLS.

Variable Dictionary

VariableRoleMeaning / CodingCoefficient Interpretation
G3OutcomeFinal grade from 0 to 19.Predicted final-grade points.
G1Numeric predictorFirst-period grade.Expected G3 change for one higher G1 point.
G2Numeric predictorSecond-period grade.Expected G3 change for one higher G2 point.
studytimeNumeric predictorWeekly study-time category.Expected G3 change for one higher category.
failuresNumeric predictorPrevious class failures.Expected G3 change for one additional failure.
absencesNumeric predictorSchool absence count.Expected G3 change for one additional absence.
ageNumeric predictorStudent age.Expected G3 change for one additional year.
MeduNumeric predictorMother’s education level.Expected G3 change for one higher education level.
school_MSDummy predictor1 = MS; 0 = GP reference.Adjusted MS versus GP difference.
sex_MDummy predictor1 = M; 0 = F reference.Adjusted M versus F difference.
address_UDummy predictor1 = urban; 0 = rural reference.Adjusted urban versus rural difference.

Error-Variance Variables

OLS fitted values, OLS residuals, squared residuals, predicted log variance, estimated variance and inverse estimated variance weights are derived quantities. They are not substantive predictors in the main G3 equation; they define the FGLS covariance correction.

OLS Diagnostic Results That Motivate GLS

DiagnosticPython StatisticPython p-valueR StatisticR p-valueInterpretation
Breusch–Pagan LM34.40860.00015719.09680.0000124Strong evidence of nonconstant variance.
Breusch–Pagan F3.57190.0001Not separately reportedSupports the same heteroscedasticity conclusion.
White LM106.87970.00034820.03660.0000446General heteroscedasticity pattern is present.
White F1.86340.0001Not separately reportedSupports rejection of constant variance.
Durbin–Watson1.8723Not applicable1.8723Not applicableClose to 2; weak first-order residual dependence.
Jarque–Bera9864.1705< .0019795.7819< .001Residuals strongly deviate from normality.

Python and R use somewhat different auxiliary-test implementations, so the exact Breusch–Pagan and White statistics differ. Both software packages produce the same decision: heteroscedasticity is statistically clear. The Durbin–Watson statistic and GLSAR estimate show that serial dependence is not the principal problem.

Normality context: Jarque–Bera significance is not the reason for using GLS. GLS corrects the covariance structure. Residual nonnormality may still affect small-sample inference and tail prediction, but it is a separate issue.

OLS, Feasible GLS, Diagonal GLS and GLSAR Model Comparison

Python Model Comparison

ModelNdf Modeldf ResidualAdjusted R²AICBICRMSEMAEDW
OLS baseline649106380.85240.85012143.34002192.56981.24030.77911.8723
Feasible GLS / WLS649106380.86950.86742037.00532086.23511.24230.77211.8716
GLS diagonal covariance649106380.86950.86742037.00212086.23181.24230.77211.8716
GLSAR AR(1) comparison649106370.85000.84772137.72072186.93351.24050.77861.8668

Feasible WLS and diagonal GLS are numerically equivalent for the estimated diagonal covariance matrix. Their weighted R² exceeds OLS by approximately 0.0171. MAE improves by 0.0070, but RMSE increases by 0.0020. The results therefore support the variance correction without claiming that every unweighted error criterion improves.

R Model Comparison

R reproduces OLS R² = 0.8523746, adjusted R² = 0.8500607, RMSE = 1.240328 and MAE = 0.7790877. Feasible GLS/WLS has R² = 0.8694957, adjusted R² = 0.8674502, RMSE = 1.242313 and MAE = 0.7721485.

R reports AIC/BIC as 2145.340/2199.045 for OLS and 2039.003/2092.708 for FGLS. These differ from Python by additive likelihood-convention constants. Compare AIC and BIC within the same software, not across different likelihood definitions.

OLS and Feasible GLS Coefficient Interpretation

Exact OLS Coefficients from the Reports

TermOLS BSEtp-value95% CIDecision
Intercept-0.16460.7837-0.21010.8337-1.7035 to 1.3742Not significant
school: MS vs GP-0.17880.1191-1.50150.1337-0.4126 to 0.0550Not significant
sex: M vs F-0.19170.1045-1.83490.0670-0.3969 to 0.0135Not significant
address: U vs R0.10820.11510.94020.3475-0.1178 to 0.3343Not significant
G10.13370.03683.63440.00030.0615 to 0.2060Significant positive
G20.88140.034225.7364< .0010.8142 to 0.9487Significant positive
studytime0.06410.06331.01210.3119-0.0602 to 0.1884Not significant
failures-0.23150.0950-2.43770.0151-0.4179 to -0.0450Significant negative
absences0.01770.01121.58820.1127-0.0042 to 0.0397Not significant
age0.02220.04360.50890.6110-0.0634 to 0.1078Not significant
Medu-0.03790.0467-0.81120.4176-0.1297 to 0.0539Not significant

FGLS Coefficient Changes Shown in Chart 6

The PDF coefficient table is truncated after the FGLS intercept, which is exactly -0.3473, SE = 0.7111, p = 0.6254, 95% CI [-1.7436, 1.0490]. The comparison chart displays the remaining plotted FGLS coefficients graphically. Approximate bar values are school_MS ≈ -0.15, sex_M ≈ -0.16, address_U ≈ 0.08, G1 ≈ 0.14, G2 ≈ 0.86, studytime ≈ 0.05, failures ≈ -0.29 and absences ≈ 0.01.

The substantive pattern remains stable: G2 is the dominant positive coefficient, G1 is positive, and failures is negative. Weighting changes the magnitudes modestly, particularly making the failures coefficient more negative. Exact FGLS coefficient reporting should use the generated CSV table rather than reverse-reading the bar chart.

Transparency rule: approximate values inferred from an unlabeled coefficient bar chart are identified as approximate. Do not present them as exact four-decimal software output.

How Generalized Least Squares Weights Work

The full model assigns larger weights to observations with smaller estimated error variance. A weight twice as large does not mean an observation is twice as important substantively. It means the fitted covariance model treats it as having greater conditional precision.

Estimated VarianceInverse WeightInfluence in FGLS
SmallLargeObservation contributes more to the weighted objective.
ModerateModerateObservation receives typical influence.
LargeSmallObservation is down-weighted because its outcome is less precise.

Weights should be normalized only when needed for interpretation or software stability. Multiplying every weight by the same constant does not change WLS coefficient estimates, although it may affect reported likelihood scales.

Weight Diagnostics

Inspect minimum, maximum, quantiles and relationships with fitted values and predictors. Extremely large weights can cause a few observations to dominate. Extremely small weights can effectively remove observations from coefficient estimation. Trimming weights changes the estimand and requires justification.

Generalized Least Squares Regression Assumptions

AssumptionMeaningDiagnosticRisk if Violated
Correct conditional meanE(y|X) is represented appropriately.Residual patterns, theory, transformations and RESET.Both OLS and GLS coefficients can be biased.
ExogeneityE(ε|X) = 0.Research design and temporal ordering.GLS does not fix endogeneity.
Covariance model reasonably specifiedEstimated Ω reflects the main variance/dependence pattern.Post-GLS residual plots and sensitivity models.Efficiency and standard errors may be misleading.
Positive weights / positive-definite ΩThe transformation is mathematically valid.Check finite positive estimated variances and matrix eigenvalues.Model cannot be fitted stably.
No perfect multicollinearityPredictors are not exact combinations.Multicollinearity check, VIF and tolerance.Coefficients are unidentified.
Independent observations conditional on ΩRemaining dependence is represented.Study design, residual autocorrelation and clustering.Uncertainty may remain understated.

GLS does not require raw outcomes to be normally distributed for coefficient consistency. Normality is mainly relevant to exact finite-sample inference and predictive distribution assumptions.

AdvertisementGoogle AdSense middle placement reserved here

Python Chart-by-Chart Interpretation with Real Values

Python Chart 1: G3 Outcome Distribution

Python GLS regression histogram of G3 outcome distribution
Distribution of the 649 G3 outcome values used by OLS and feasible GLS.

The analysis uses 649 G3 observations. The distribution has mean 11.9060, median 12, SD 3.2307, first quartile 10, third quartile 14, minimum 0 and maximum 19. The highest frequencies occur at G3 = 11 with 104 records, G3 = 10 with 97, G3 = 13 with 82 and G3 = 12 with 72.

The histogram is concentrated around grades 10–15 but includes 15 zero scores. These low outcomes create the long negative residual tail visible in both OLS and FGLS diagnostic plots. GLS changes weighting; it does not transform the bounded integer outcome into a normal variable.

Python Chart 2: Observed vs Fitted Values for OLS and FGLS

Python observed versus fitted G3 values for OLS and feasible GLS
Observed G3 plotted against overlapping OLS and feasible GLS fitted values.

Both fitted series track the 45-degree line closely through the middle of the grade distribution. OLS has R² = 0.8524, RMSE = 1.2403 and MAE = 0.7791. FGLS has weighted R² = 0.8695, RMSE = 1.2423 and MAE = 0.7721.

The two point clouds overlap heavily because coefficient changes are modest. Several zero outcomes receive fitted values between approximately 5 and 10, producing large negative residuals. FGLS improves the weighted objective and MAE, but it does not eliminate these unusual cases.

Python Chart 3: OLS Residuals vs Fitted Values

Python OLS residuals versus fitted values for GLS diagnostic
OLS residual pattern used to motivate the feasible GLS variance correction.

OLS residuals range visually from approximately -9.1 to 5.5. The main residual band is concentrated between about -2 and 2, but the distribution includes large negative points around fitted values 5–10. The diagonal stripes arise because observed G3 is integer valued while fitted values are continuous.

Formal tests confirm that the visible spread is not constant: Breusch–Pagan LM = 34.4086, p = 0.000157, and White LM = 106.8797, p = 0.000348. These values, rather than visual appearance alone, motivate the FGLS correction.

Python Chart 4: Feasible GLS Residuals vs Fitted Values

Python feasible GLS residuals versus fitted values
Residuals from the inverse estimated variance weighted model.

The FGLS residual plot retains a similar raw-residual range, approximately -9.1 to 5.6, because weighting does not guarantee smaller residuals for every observation. The central point band remains near zero and the same integer-outcome diagonal structure is visible.

FGLS MAE decreases from 0.7791 to 0.7721, but RMSE rises slightly from 1.2403 to 1.2423. The correct interpretation is not that every residual became smaller. The model redistributes influence according to estimated precision and improves the weighted likelihood and average absolute error.

Python Chart 5: OLS vs FGLS Absolute Residual Spread

Python boxplot comparing OLS and FGLS absolute residual spread
Absolute residual distributions for OLS and feasible GLS.

The two boxplots have very similar central ranges and extreme outliers reaching above 9. OLS MAE is 0.7791, while FGLS MAE is 0.7721, an improvement of about 0.0070. The boxplot median appears slightly lower for FGLS, but the difference is small.

The chart is a useful warning against exaggerated claims. GLS is strongly justified by the variance diagnostics, yet the unweighted absolute-residual distribution changes only modestly. Efficiency correction and dramatic point-prediction improvement are not the same thing.

Python Chart 6: OLS vs Feasible GLS Coefficient Comparison

Python bar chart comparing OLS and feasible GLS coefficients
Coefficient estimates for the main plotted terms under OLS and FGLS.

OLS plotted coefficients are school_MS = -0.1788, sex_M = -0.1917, address_U = 0.1082, G1 = 0.1337, G2 = 0.8814, studytime = 0.0641, failures = -0.2315 and absences = 0.0177. The FGLS bars remain close but shift approximately to -0.15, -0.16, 0.08, 0.14, 0.86, 0.05, -0.29 and 0.01.

G2 remains the dominant effect under both estimators. The largest visible change occurs for failures, which becomes more negative under FGLS. Exact FGLS coefficients should be copied from the output CSV because the PDF coefficient table is truncated.

Python Chart 7: Feasible GLS Weight Distribution

Python histogram of feasible GLS inverse estimated variance weights
Distribution of inverse estimated error-variance weights across 649 observations.

Weights span approximately 1.9 to 5.1. The highest-frequency bin is around 2.8–3.0 with slightly more than 80 observations. Other dense regions occur around 2.5–2.8 and 3.1–3.5. Relatively few records receive weights above 4.5.

The moderate range indicates that the variance model changes influence without creating extremely dominant observations. Higher weights correspond to lower predicted error variance; lower weights correspond to noisier observations.

Python Chart 8: OLS Diagnostic p-Values

Python GLS diagnostic p values for Breusch Pagan White and Jarque Bera tests
Diagnostic p-values compared with the alpha = .05 reference line.

Every displayed p-value is far left of the .05 reference line. Python reports Breusch–Pagan LM p = 0.000157, Breusch–Pagan F p near 0.0001, White LM p = 0.000348, White F p near 0.0001 and Jarque–Bera p < 0.0001.

The heteroscedasticity tests support GLS. The Jarque–Bera result supports residual-distribution caution but does not determine the choice between OLS and GLS. Durbin–Watson is excluded because it is a statistic rather than a p-value.

Python Chart 9: Observed and Fitted Means by G1

Python observed OLS and FGLS fitted G3 means by G1
Grouped observed, OLS fitted and FGLS fitted means across G1 values.

At G1 = 0, the observed mean is approximately 11.0, while OLS and FGLS means are near 10.1. At G1 = 5, the observed mean is approximately 2.6, whereas both fitted means are near 4.0. From G1 = 8 through 17, the three lines generally track closely.

At G1 = 18, the fitted means rise to about 18.5, compared with an observed mean near 18.0. At G1 = 19, the observed mean is about 18.0 and fitted means are about 17.7–17.8. The chart shows excellent central calibration with larger discrepancies at sparse extreme G1 levels.

R Charts, Tables and Independent Validation

The R report independently reproduces the 649-row formula, heteroscedasticity decision, model comparison and nine chart topics. The supplied list repeats the same chart URLs, so the R section uses those confirmed assets while interpreting the R-specific numerical results.

Open the complete Generalized Least Squares Regression R report PDF

R Chart 1: Outcome Distribution Validation

R validation of G3 outcome distribution for GLS regression
R validation of the 649-observation G3 distribution.

R uses the same outcome mean 11.9060, median 12, SD 3.2307 and range 0–19. This confirms that OLS and FGLS differences are estimator differences rather than changes in the analyzed sample.

R Chart 2: Observed vs Fitted Validation

R validation of observed versus fitted OLS and FGLS values
R validation of the overlapping OLS and feasible GLS fitted patterns.

R reports OLS RMSE = 1.240328 and MAE = 0.7790877. FGLS reports RMSE = 1.242313 and MAE = 0.7721485. The same conclusion holds: average absolute error improves slightly, while RMSE changes minimally in the opposite direction.

R Chart 3: OLS Residual Diagnostic Validation

R validation of OLS residuals versus fitted values
R interpretation of the OLS residual pattern before variance weighting.

R’s Breusch–Pagan LM statistic is 19.0968 with p = 1.24 × 10⁻⁵. The White statistic is 20.0366 with p = 4.46 × 10⁻⁵. Both validate the nonconstant variance decision despite differing from Python’s auxiliary-test statistics.

R Chart 4: FGLS Residual Diagnostic Validation

R validation of feasible GLS residuals versus fitted values
R interpretation of residuals after inverse-variance weighting.

The weighted fit modifies the coefficient solution and influence structure, but raw residual outliers remain. Post-FGLS residual checks should evaluate whether the variance relationship is reduced in transformed or standardized residuals, not only whether every raw residual shrinks.

R Chart 5: Residual Spread Validation

R validation of OLS and FGLS absolute residual spread
R validation of the modest MAE improvement and similar extreme residuals.

R MAE decreases by approximately 0.00694. The boxplots remain similar because the strongest correction is to observation influence and likelihood, not a universal compression of raw residual magnitudes.

R Chart 6: Coefficient Comparison Validation

R validation of OLS and FGLS coefficient comparison
R validation of stable coefficient signs and modest weighting-related shifts.

R reproduces exact OLS coefficients G1 = 0.133732, G2 = 0.881441 and failures = -0.231465. The FGLS comparison preserves the dominant G2 effect and negative failures direction.

R Chart 7: Weight Distribution Validation

R validation of feasible GLS inverse variance weight distribution
R validation of moderate inverse estimated variance weights.

The distribution concentrates between approximately 2 and 4, with a thinner upper tail to about 5.1. This indicates meaningful but not extreme reweighting.

R Chart 8: Diagnostic p-Value Validation

R validation of GLS diagnostic p values
R diagnostic decisions compared with the .05 threshold.

R’s heteroscedasticity p-values are even smaller than Python’s: 0.0000124 and 0.0000446. Both are far below .05. Jarque–Bera is effectively zero, confirming nonnormal residual tails.

R Chart 9: Observed and Fitted Means by G1 Validation

R validation of observed OLS and FGLS fitted means by G1
R validation of grouped calibration across the main prior-grade predictor.

The observed and fitted mean lines nearly overlap for most G1 values, with the largest discrepancy near G1 = 5. R therefore confirms strong central fit and sparse-level instability at the extremes.

R validation conclusion: R reproduces the same model hierarchy and practical conclusion as Python. FGLS is justified by heteroscedasticity and improves the weighted fit and MAE, while the basic academic coefficient pattern remains stable.

SPSS Output Status and Corrected GLS Workflow

Open the currently supplied SPSS PDF

Current file is invalid: the one-page PDF contains only an SPSS warning that EMBEDBOOKMARKS is not a recognized subcommand and that command execution stopped. It contains no regression model summary, ANOVA table, coefficient table, saved residuals or weight diagnostics.

Required SPSS Export Correction

OUTPUT EXPORT
  /CONTENTS EXPORT=ALL LAYERS=PRINTSETTING MODELVIEWS=PRINTSETTING
  /PDF DOCUMENTFILE=
'D:\DATA ANALYSIS\H Regression Tests and Models\Generalized Least Squares Regression\SPSS_Output\pdf\generalized_least_squares_regression_spss_report.pdf'.

Do not use EMBEDBOOKMARKS or EMBEDFONTS as separate OUTPUT EXPORT subcommands. Regenerate the Viewer output and PDF after removing them.

Practical SPSS FGLS Procedure

SPSS does not provide the same one-command heteroscedastic FGLS workflow as the Python/R scripts. A compatible approach is:

  1. Fit baseline REGRESSION and save predicted values and residuals.
  2. Compute squared residuals and log squared residuals with a small constant.
  3. Regress log residual squared on absolute fitted value or selected variance predictors.
  4. Save predicted log variance.
  5. Compute estimated variance as EXP(predicted log variance).
  6. Compute inverse variance weight.
  7. Use WLS or weighted REGRESSION with the estimated weight.
  8. Save a valid SAV, SPV and PDF.
Expected SPSS OutputRequired ContentCurrent Status
Case processingN = 649 complete observations.Missing
OLS model summaryR², adjusted R², standard error.Missing
OLS coefficientsB, SE, t, Sig., 95% CI.Missing
Variance modelPredicted log variance and weights.Missing
Weighted regressionFGLS coefficient table and model summary.Missing
DiagnosticsResidual charts, BP/White equivalent context and DW.Missing
PDF exportCompatibility-safe OUTPUT EXPORT.Failed due obsolete subcommand

Excel Worked Generalized Least Squares Results

The Excel workbook is a separate 10-row formula demonstration. It does not reproduce the 649-row model coefficients. It explains how OLS residuals lead to estimated variances, inverse weights and FGLS fitted values.

Excel Demonstration Summary

MetricExcel ValueInterpretation
Rows10Small teaching example.
OLS RMSE0.673795Root mean squared OLS residual.
FGLS RMSE0.459347Lower demonstration error after weighting.
Mean OLS absolute residual0.620000Average unweighted absolute OLS error.
Mean FGLS absolute residual0.410000Average absolute FGLS error.
Mean GLS weight0.305738Average inverse estimated variance.
Largest weight0.406570Most precise demonstration observation.
Smallest weight0.173774Least precise demonstration observation.

Ten-Row Formula Demonstration

For row 1, observed Y = 10, OLS fitted = 10.4 and OLS residual = -0.4. The predicted log variance is 1.15, estimated variance is 3.1582, inverse weight is 0.3166, FGLS fitted = 10.2 and FGLS residual = -0.2.

Row 7 has the smallest weight, approximately 0.1738, because its estimated variance is 5.7546. Row 8 has the largest weight, approximately 0.4066, because its estimated variance is 2.4596. This directly demonstrates inverse-variance weighting.

Excel Formula Patterns

QuantityFormula PatternPurpose
OLS residual=ObservedY-OLSFittedInitial model error.
Residual squared=OLSResidual^2Variance proxy.
Log residual squared=LN(ResidualSquared+0.000001)Variance-model outcome.
Estimated variance=EXP(PredictedLogVariance)Positive variance estimate.
GLS weight=1/EstimatedVarianceInverse estimated precision weight.
FGLS residual=ObservedY-FGLSFittedWeighted-model error.
RMSE=SQRT(AVERAGE(ResidualSquaredRange))Root mean squared error.
MAE=AVERAGE(AbsoluteResidualRange)Mean absolute error.

Do not mix workbook and full-data results: Excel RMSE 0.4593 describes the 10-row demonstration. The 649-row FGLS RMSE is 1.2423.

Python, R, SPSS and Excel GLS Workflows

SoftwareMain WorkflowOutputsStatus
Pythonstatsmodels OLS, variance regression, WLS, GLS diagonal covariance and GLSAR.Nine charts, CSV tables, TXT report and confirmed PDF.Complete
Rlm baseline, variance model, weighted lm/FGLS and diagnostic tests.Nine charts, CSV tables, TXT report and confirmed PDF.Complete
SPSSBaseline REGRESSION, saved residuals, variance model, computed weights and weighted regression.SAV, SPV and PDF expected.PDF export failed and must be regenerated
ExcelTen-row residual, variance, weight and fitted-value formulas.Worked explanatory workbook.Complete teaching example

Generalized Least Squares Regression Code

Python Code

import numpy as np
import pandas as pd
import statsmodels.api as sm
import statsmodels.formula.api as smf
from statsmodels.stats.diagnostic import het_breuschpagan, het_white
from statsmodels.stats.stattools import durbin_watson

df = pd.read_csv("dataset.csv")

formula = (
    "G3 ~ G1 + G2 + studytime + failures + absences + age + Medu"
    " + C(school) + C(sex) + C(address)"
)

work = df[
    ["G3", "G1", "G2", "studytime", "failures",
     "absences", "age", "Medu", "school", "sex", "address"]
].dropna().copy()

# OLS baseline
ols = smf.ols(formula, data=work).fit()
work["ols_fitted"] = ols.fittedvalues
work["ols_resid"] = ols.resid

# Heteroscedasticity tests
bp = het_breuschpagan(ols.resid, ols.model.exog)
white = het_white(ols.resid, ols.model.exog)
print("BP LM p:", bp[1])
print("White LM p:", white[1])
print("DW:", durbin_watson(ols.resid))

# Variance model
small = 1e-8
work["log_resid_sq"] = np.log(work["ols_resid"] ** 2 + small)
variance_model = smf.ols(
    "log_resid_sq ~ I(abs(ols_fitted))",
    data=work
).fit()

work["predicted_log_variance"] = variance_model.predict(work)
work["estimated_variance"] = np.exp(work["predicted_log_variance"])
work["gls_weight"] = 1.0 / work["estimated_variance"]

# Feasible GLS as WLS
fgls = smf.wls(
    formula,
    data=work,
    weights=work["gls_weight"]
).fit()

# Equivalent diagonal GLS
sigma = np.diag(work["estimated_variance"].to_numpy())
gls = sm.GLS(
    ols.model.endog,
    ols.model.exog,
    sigma=sigma
).fit()

print(fgls.summary())
print(gls.summary())

R Code

df <- read.csv("dataset.csv", stringsAsFactors = TRUE)

formula_gls <- G3 ~ G1 + G2 + studytime + failures +
  absences + age + Medu + school + sex + address

work <- na.omit(df[
  c("G3", "G1", "G2", "studytime", "failures",
    "absences", "age", "Medu", "school", "sex", "address")
])

ols <- lm(formula_gls, data = work)
work$ols_fitted <- fitted(ols)
work$ols_residual <- residuals(ols)

small <- 1e-8
work$log_resid_sq <- log(work$ols_residual^2 + small)

variance_model <- lm(
  log_resid_sq ~ abs(ols_fitted),
  data = work
)

work$predicted_log_variance <- predict(variance_model)
work$estimated_variance <- exp(work$predicted_log_variance)
work$gls_weight <- 1 / work$estimated_variance

fgls <- lm(
  formula_gls,
  data = work,
  weights = gls_weight
)

summary(ols)
summary(fgls)

library(lmtest)
library(sandwich)
bptest(ols)
dwtest(ols)

SPSS Compatibility-Safe Syntax Pattern

* Baseline OLS.
REGRESSION
  /DEPENDENT G3
  /METHOD=ENTER G1 G2 studytime failures absences age Medu
    school_MS sex_M address_U
  /STATISTICS COEFF OUTS R ANOVA CI(95) COLLIN TOL
  /SAVE PRED(ols_pred) RESID(ols_resid).

COMPUTE resid_sq = ols_resid**2.
COMPUTE log_resid_sq = LN(resid_sq + .000001).
COMPUTE abs_ols_pred = ABS(ols_pred).
EXECUTE.

* Variance model.
REGRESSION
  /DEPENDENT log_resid_sq
  /METHOD=ENTER abs_ols_pred
  /SAVE PRED(pred_log_variance).

COMPUTE estimated_variance = EXP(pred_log_variance).
COMPUTE gls_weight = 1 / estimated_variance.
EXECUTE.

WEIGHT BY gls_weight.

REGRESSION
  /DEPENDENT G3
  /METHOD=ENTER G1 G2 studytime failures absences age Medu
    school_MS sex_M address_U
  /STATISTICS COEFF OUTS R ANOVA CI(95) COLLIN TOL
  /SAVE PRED(fgls_pred) RESID(fgls_resid).

WEIGHT OFF.

OUTPUT SAVE OUTFILE=
'D:\DATA ANALYSIS\H Regression Tests and Models\Generalized Least Squares Regression\SPSS_Output\spv\generalized_least_squares_regression_spss_output.spv'.

OUTPUT EXPORT
  /CONTENTS EXPORT=ALL LAYERS=PRINTSETTING MODELVIEWS=PRINTSETTING
  /PDF DOCUMENTFILE=
'D:\DATA ANALYSIS\H Regression Tests and Models\Generalized Least Squares Regression\SPSS_Output\pdf\generalized_least_squares_regression_spss_report.pdf'.

Excel Formula Pattern

OLS residual:
=ObservedY-OLSFitted

Residual squared:
=OLSResidual^2

Log residual squared:
=LN(ResidualSquared+0.000001)

Estimated variance:
=EXP(PredictedLogVariance)

GLS inverse variance weight:
=1/EstimatedVariance

Weighted least squares objective:
=SUMPRODUCT(WeightRange,ResidualRange,ResidualRange)

Generalized Least Squares vs Heteroscedasticity-Robust Standard Errors

FeatureGLS / FGLSOLS with Robust SE
Coefficient estimatesCan change because observations are reweighted/transformed.Remain equal to OLS coefficients.
Covariance modelMust be specified or estimated.No explicit variance model required.
EfficiencyCan improve if Ω is well specified.Consistent uncertainty but not necessarily efficient coefficients.
Misspecification riskIncorrect weights can harm efficiency and inference.More robust to unknown heteroscedasticity.
PredictionWeighted fit may change predictions.Predictions remain OLS predictions.
Recommended useStrong, interpretable variance/dependence structure.Heteroscedasticity is present but variance model is uncertain.

A useful sensitivity analysis reports OLS with HC3 standard errors alongside FGLS. If substantive conclusions change dramatically, investigate whether a small set of weights or variance-model assumptions drives the difference.

Prediction, Fit and Model Selection in GLS

Weighted R² is calculated using the weighted residual objective and should not automatically be compared with unweighted OLS R² as though they were the same quantity. AIC and BIC also depend on likelihood conventions and weight scaling.

How to Interpret This Model Comparison

MetricOLSFGLSConclusion
Weighted / reported R²0.85240.8695FGLS better under its weighted objective.
Adjusted R²0.85010.8674FGLS weighted fit remains higher after df adjustment.
MAE0.77910.7721Small FGLS improvement.
RMSE1.24031.2423Small OLS advantage on unweighted squared error.
AIC within Python2143.34002037.0053FGLS likelihood fit is much better under Python’s implementation.
Durbin–Watson1.87231.8716Weighting does not materially change weak serial dependence.

For prediction, evaluate cross-validated unweighted error, weighted error and calibration across variance strata. In-sample weighted AIC improvement is not enough to establish superior out-of-sample prediction.

APA-Style Reporting for Generalized Least Squares Regression

Full Python Result

A feasible generalized least squares regression was fitted to predict G3 from G1, G2, studytime, failures, absences, age, maternal education, school, sex and address using 649 complete observations. OLS residual diagnostics indicated heteroscedasticity, Breusch–Pagan LM = 34.41, p < .001, and White LM = 106.88, p < .001. Durbin–Watson was 1.87, indicating little first-order residual dependence. Inverse estimated error-variance weights were derived from a log residual-variance model. The feasible GLS model reported R² = .870, adjusted R² = .867, MAE = .772 and RMSE = 1.242, compared with OLS R² = .852, adjusted R² = .850, MAE = .779 and RMSE = 1.240.

Coefficient Summary

The OLS and FGLS coefficient patterns were substantively similar. G2 remained the dominant positive predictor, OLS B = 0.881, 95% CI [0.814, 0.949], p < .001. G1 was positive, B = 0.134, 95% CI [0.061, 0.206], p < .001, and failures was negative, B = -0.231, 95% CI [-0.418, -0.045], p = .015. The FGLS coefficient-comparison chart showed modest weighting-related changes rather than sign reversals.

SPSS Qualification

The supplied SPSS PDF could not be interpreted because export stopped after an unrecognized EMBEDBOOKMARKS subcommand. SPSS results should be reported only after regenerating the output with the compatibility-safe OUTPUT EXPORT syntax.

Generalized Least Squares Reporting Checklist

Reporting ItemRequired DetailWhy It Matters
OutcomeName, units and range.Defines coefficient scale.
PredictorsNumeric variables, factors and reference levels.Supports interpretation and replication.
OLS baselineCoefficients, fit and residual diagnostics.Shows why GLS was considered.
Covariance problemHeteroscedasticity, autocorrelation or clustering.Defines the correction target.
Variance/correlation modelExact equation or covariance structure.FGLS depends on this specification.
Weight constructionInverse known or estimated variance.Explains differential influence.
Weight rangeMinimum, maximum and quantiles.Identifies influential precision weights.
GLS coefficient tableB, SE, statistic, p and 95% CI.Primary inferential result.
Model comparisonR² type, AIC/BIC, MAE and RMSE.Prevents metric cherry-picking.
Post-fit diagnosticsWeighted/standardized residual checks.Evaluates covariance correction quality.
Sensitivity analysisOLS robust SE and alternative variance models.Tests dependence on FGLS assumptions.
Software detailsPackage, tie/likelihood convention and weight scaling.Explains implementation differences.

Common Generalized Least Squares Mistakes

MistakeWhy It Is WrongBetter Practice
Using GLS only because OLS residuals are nonnormalGLS targets covariance, not distribution shape alone.Identify heteroscedasticity or dependence explicitly.
Treating arbitrary case weights as inverse variancesThe estimator no longer has the intended precision interpretation.Document where every weight comes from.
Claiming FGLS improves every prediction metricHere MAE improves but RMSE slightly worsens.Report all relevant metrics.
Comparing AIC across Python and R directlyLikelihood constants and weight conventions differ.Compare candidate models within the same software.
Ignoring weight extremesA few high weights can dominate coefficients.Inspect distribution and sensitivity to trimming.
Using FGLS without post-fit diagnosticsThe estimated variance model may remain inadequate.Review weighted and standardized residuals.
Assuming GLS fixes endogeneityIt changes error covariance, not predictor exogeneity.Use design or instrumental-variable methods.
Reporting the current SPSS PDF as outputIt contains only an export error.Regenerate SAV/SPV/PDF with safe syntax.
Using the GEE PDF in the GLS postGEE is a different correlated-data estimator.Keep Generalized Estimating Equations in its own article.

Generalized Least Squares Troubleshooting Guide

Weights Are Infinite or Missing

Predicted variance may be zero, negative before exponentiation, missing or numerically extreme. Use a log-variance model, verify finite fitted values and apply reasonable numerical bounds only with documented justification.

FGLS Coefficients Change Dramatically

Inspect whether a small number of observations receive very large weights. Compare with OLS HC3 results and refit after checking data errors. Dramatic changes may indicate variance-model misspecification.

FGLS R Squared Is Higher but RMSE Is Worse

This is possible because weighted R² optimizes a different objective than unweighted RMSE. In this analysis, FGLS adjusted R² rises to .8674 while RMSE changes from 1.2403 to 1.2423.

Python and R AIC Values Differ by a Constant

Different implementations include different likelihood constants or weight normalizations. Compare differences between OLS and FGLS inside Python or inside R, not raw values across languages.

Breusch–Pagan and White Statistics Differ Across Software

The auxiliary regressions and degrees of freedom may differ. Focus on whether the same null decision is reached. Here every implementation strongly rejects constant variance.

Durbin–Watson Is Near 2 but GLSAR Is Included

GLSAR is a comparison model. With ρ = 0.0661 and DW = 1.8723, autocorrelation is weak; the main correction remains heteroscedastic FGLS.

SPSS PDF Contains Only EMBEDBOOKMARKS Error

Remove the unsupported subcommand, rerun the entire syntax, save the Viewer file and export with the safe PDF block. Do not merely rename the failed one-page PDF.

FGLS Residual Plot Still Has Outliers

GLS does not guarantee elimination of unusual outcomes. Check studentized residuals, Cook’s distance, Mahalanobis distance and outlier detection.

Variance Model Produces No Improvement

The chosen variance predictor may not describe heteroscedasticity. Consider multiple variance predictors, group-specific variances, smooth functions or robust OLS instead of forcing FGLS.

Downloads and Resources

AdvertisementGoogle AdSense bottom placement reserved here

FAQs About Generalized Least Squares Regression

What is generalized least squares regression?

GLS is linear regression that estimates coefficients while accounting for unequal error variances or correlations through an error covariance matrix.

What is feasible generalized least squares?

FGLS estimates the unknown error covariance from preliminary residuals and then fits GLS using the estimated covariance.

Why was GLS used in this example?

Breusch–Pagan and White tests strongly rejected constant OLS error variance, while Durbin–Watson showed only weak serial dependence.

What variables were used?

G3 was predicted from G1, G2, studytime, failures, absences, age, Medu, school, sex and address.

What was the FGLS adjusted R squared?

The Python report gives adjusted R² = 0.8674, compared with 0.8501 for OLS.

Did FGLS improve prediction?

MAE improved slightly from 0.7791 to 0.7721, while RMSE changed slightly from 1.2403 to 1.2423. Improvement depends on the metric and weighting objective.

What is the GLS weight?

The workflow uses the inverse of estimated observation-specific error variance. Higher weights indicate greater estimated precision.

Is WLS the same as GLS?

WLS is a special case of GLS with a diagonal covariance matrix. General GLS can also model correlations between errors.

Is GLS better than robust standard errors?

GLS can be more efficient when the covariance model is correct. Robust OLS requires fewer covariance assumptions and keeps OLS coefficients.

Why do Python and R AIC values differ?

The packages use different likelihood constants or weight scaling. Model comparisons should be made within each software.

Was autocorrelation a major problem?

No. Durbin–Watson was 1.8723 and the GLSAR rho estimate was only 0.0661.

Can GLS be done in SPSS?

Yes through a saved-residual variance model and weighted regression, but the supplied PDF failed during export and must be regenerated.

What is wrong with the supplied SPSS PDF?

It contains only an error saying EMBEDBOOKMARKS is not recognized. No regression results were exported.

Can GLS be calculated in Excel?

Excel can demonstrate residual variance, inverse weights and weighted fitted values. Python or R is preferable for complete coefficient estimation and diagnostics.

Does GLS fix endogeneity?

No. GLS corrects the error covariance structure. It does not make endogenous predictors exogenous.

Final Generalized Least Squares Regression Conclusion

The 649-row analysis provides strong evidence that OLS errors are heteroscedastic. Feasible GLS uses inverse estimated error-variance weights and increases the reported weighted adjusted R² from 0.8501 to 0.8674. It lowers MAE from 0.7791 to 0.7721 and substantially improves within-software likelihood criteria, while unweighted RMSE remains nearly unchanged.

G2 remains the dominant academic predictor, G1 remains positive, and failures remains negative. The correction changes coefficient magnitude and influence modestly rather than reversing the substantive model.

The correct final report should state the variance model, explain the weight construction, distinguish weighted from unweighted fit measures, provide sensitivity analysis against robust OLS and regenerate the SPSS output with compatibility-safe PDF syntax before publishing SPSS-specific numerical conclusions.

Need help applying this to your own data?

Salar Cafe can help interpret output, clean datasets, review assumptions, build dashboards and explain statistical results ethically.

Need help interpreting your data analysis results?

Contact Salar Cafe
Engr. Muhammad Yar Saqib author profile photo

Engr. Muhammad Yar Saqib

WhatsApp Get Data Analysis Help