Methods in Political Science

Multiple Regression Analysis

Sirus Dehdari

Department of Political Science, Stockholm University

Last week:

• Is there a relationship between two variables?
• Example: is there a relationship between indoor temperature and a household’s monthly income?
• How well does the model fit our data?
• Statistical inference: are we confident that a relationship actually exists in the population?
• Confidence interval for the estimated slope coefficient, \(b\)

Third week’s learning objectives

• Understand what multiple regression analysis is and why it’s used
• How to interpret the slope coefficient in a multiple regression model
• Causal inference
• “Economic” significance
• Chapter 5.2 in Österman and Folke

Remember: bivariate regression model

• Last week we talked about bivariate regression models
• We used one independent variable to explain the variation in the dependent variable
• Can’t we use more than one independent variable to explain the variation in the dependent variable?
• In other words: can’t the variation in the dependent variable depend on more than one independent variable?
• A regression model with more than one independent variable is called a multiple regression model

Including more independent variables

• Remember the equation for a bivariate regression model:

\[Y_i = a + b_1X_{1i} + e_i\]

• Note the index 1 on \(b\) and \(X\): they are indicators for the first variable
• We can include more independent variables

\[Y_i = a + b_1X_{1i} + b_2X_{2i} + e_i\]

• Now we’ve included a second independent variable, \(X_2\), in the model

Multiple regression model

• If there is a relationship between \(X_2\) and \(Y\) (i.e. \(b_2\neq 0\)), the multiple regression model will explain even more of the variation in \(Y\)
• There may be even more variables that help us explain the variation in \(Y\)

\[Y_i = a + b_1X_{1i} + b_2X_{2i} + b_3X_{3i} + \ldots + b_kX_{ki} + \ldots + e_i\]

• Here we’ve included a total of \(k\) different independent variables
• But why include more than one variable at all?

Why multiple regression models?

• The two most common reasons we add more independent variables:
  • Explain more of the variation in the dependent variable \(Y\)
  • Rule out that the relationship between \(Y\) and \(X_1\) is affected by some other variable
• In research where we’re interested in the outcome (the dependent variable), we often want to explain as much of its variation as possible
• Example: What determines voters’ propensity to vote?
• Variables that can explain turnout: income, education level, citizenship, marital status, age, etc.

Explaining the variation in the outcome

• By explaining the variation in the outcome, we believe we understand this specific phenomenon
• If there’s still unexplained variation in, say, turnout, we need to develop new or better theories
• Example: many municipalities want to increase early voting
• A regression model that explains a large share of the variation in the propensity to vote early can help municipalities promote early voting

Can we explain too much?

• It’s usually valuable to explain as much as possible of the variation in the dependent variable
• However, variables shouldn’t be included at random: there should be a reason for including a variable
• We won’t go into the technical details of why we should avoid adding too many independent variables
• A rule of thumb: only add variables you believe are related to the dependent variable

Causal relationships

• So far we’ve only considered the relationship between \(Y\) and \(X\) as nothing more than a correlation
• But we often want to understand whether there’s a causal relationship between these variables
• Is it \(X\) that causes the change in \(Y\)?
• This is the main reason we estimate multiple regression models
• \(X\) can cause \(Y\), but there may be other variables that affect both \(X\) and \(Y\)

A third factor: confounder

• \(X\) causes \(Y\), but the confounder (also called a confounding factor) \(c\) causes both \(X\) and \(Y\)

// One-time (non-reactive) global listener: on mobile, rotating the
// screen fires "orientationchange" while iOS Safari is still mid-way
// through animating the address bar/viewport, so reveal.js's own
// resize handler picks up transitional (wrong) dimensions and mis-
// scales the slide -- which is what made every interactive figure
// disappear until a full reload. Re-running reveal.js's own layout
// pass a moment after rotation, once the viewport has settled, fixes
// this without touching any figure's own code.
orientationRelayout = {
  window.addEventListener("orientationchange", () => {
    setTimeout(() => {
      if (typeof Reveal !== "undefined" && Reveal.layout) Reveal.layout();
    }, 400);
  });
  return null;
}

// Each arrow's own explanatory text, keyed so the highlight/tooltip
// logic below can tell them apart. Uses the same example as the
// "Example: political participation" slide right after this one, so
// the DAG sets up that example instead of introducing an unrelated
// one.
dagLines = [
  { key: "cx", x1: 100, y1: 460, x2: 100, y2: 65, text: "c affects X: e.g. high unemployment affects the likelihood of moving away from the region" },
  { key: "xy", x1: 100, y1: 65, x2: 575, y2: 178, text: "X affects Y: e.g. regional out-migration affects support for populist parties" },
  { key: "cy", x1: 100, y1: 460, x2: 575, y2: 196, text: "c affects Y: e.g. high unemployment directly affects support for populist parties" }
]

function dagDistToSegment(px, py, x1, y1, x2, y2) {
  const dx = x2 - x1, dy = y2 - y1;
  const lenSq = dx * dx + dy * dy;
  let t = lenSq === 0 ? 0 : ((px - x1) * dx + (py - y1) * dy) / lenSq;
  t = Math.max(0, Math.min(1, t));
  const cx = x1 + t * dx, cy = y1 + t * dy;
  return Math.hypot(px - cx, py - cy);
}

// Simple greedy word-wrap so the (fairly long) explanatory sentences
// fit inside a tooltip box instead of running off the edge of the
// canvas.
function dagWrapText(text, maxChars) {
  const words = text.split(" ");
  const lines = [];
  let cur = "";
  for (const w of words) {
    if ((cur + " " + w).trim().length > maxChars && cur) {
      lines.push(cur);
      cur = w;
    } else {
      cur = (cur + " " + w).trim();
    }
  }
  if (cur) lines.push(cur);
  return lines;
}

// Built once (no reactive dependencies) -- hover/tap state is handled
// entirely via direct DOM manipulation (highlight + tooltip), the same
// pattern used for the line tooltips on L2's "Other possible lines"
// slide, rather than through a mutable/reactive rebuild.
dagSvg = {
  const width = 680, height = 520;
  const svg = d3.create("svg")
    .attr("viewBox", `0 0 ${width} ${height}`)
    .attr("width", width)
    .attr("height", height)
    .attr("class", "dag-svg")
    .style("touch-action", "none")
    .style("max-width", "100%")
    .style("display", "block")
    .style("margin", "0 auto");

  svg.append("defs").append("marker")
    .attr("id", "dag-arrow")
    .attr("markerWidth", 14).attr("markerHeight", 14)
    .attr("refX", 11).attr("refY", 4.5)
    .attr("orient", "auto").attr("markerUnits", "strokeWidth")
    .append("path")
    .attr("d", "M0,0 L0,9 L13,4.5 z")
    .attr("fill", "currentColor");

  const lineEls = dagLines.map(ln => {
    const sel = svg.append("line")
      .attr("x1", ln.x1).attr("y1", ln.y1)
      .attr("x2", ln.x2).attr("y2", ln.y2)
      .attr("stroke", "#333").attr("stroke-width", 3)
      .attr("marker-end", "url(#dag-arrow)");
    return { ...ln, sel };
  });

  svg.append("text").attr("x", 75).attr("y", 500).attr("font-size", 34).text("c");
  svg.append("text").attr("x", 75).attr("y", 55).attr("font-size", 34).text("X");
  svg.append("text").attr("x", 600).attr("y", 195).attr("font-size", 34).text("Y");

  const tipG = svg.append("g").attr("display", "none");
  const tipRect = tipG.append("rect")
    .attr("rx", 8).attr("fill", "white").attr("fill-opacity", 0.97)
    .attr("stroke", "#2563eb").attr("stroke-width", 2);
  const tipTextG = tipG.append("g");

  const lineHeight = 22;
  const padX = 14, padY = 14;
  const maxCharsPerLine = 30;

  function showTip(mx, my, ln) {
    lineEls.forEach(o => o.sel
      .attr("stroke", o.key === ln.key ? "#2563eb" : "#333")
      .attr("stroke-width", o.key === ln.key ? 5 : 3));

    const lines = dagWrapText(ln.text, maxCharsPerLine);
    tipTextG.selectAll("*").remove();
    const boxW = Math.max(...lines.map(l => l.length)) * 9 + padX * 2;
    const boxH = lines.length * lineHeight + padY * 2 - 6;
    let bx = mx + 14, by = my - boxH - 14;
    if (bx + boxW > width - 2) bx = mx - boxW - 14;
    if (bx < 2) bx = 2;
    if (by < 2) by = my + 14;
    if (by + boxH > height - 2) by = height - boxH - 2;

    tipRect.attr("x", bx).attr("y", by).attr("width", boxW).attr("height", boxH);
    lines.forEach((line, i) => {
      tipTextG.append("text")
        .attr("x", bx + padX).attr("y", by + padY + 6 + i * lineHeight)
        .attr("font-size", 17).attr("fill", "#1e3a8a")
        .text(line);
    });
    tipG.attr("display", null);
  }

  function hideTip() {
    tipG.attr("display", "none");
    lineEls.forEach(o => o.sel.attr("stroke", "#333").attr("stroke-width", 3));
  }

  // Priority order = dagLines order: where two arrows' hit areas
  // overlap (they share endpoints at c, X, and Y), the first match
  // wins.
  const hitRadius = 22;
  function findLine(mx, my) {
    for (const ln of lineEls) {
      if (dagDistToSegment(mx, my, ln.x1, ln.y1, ln.x2, ln.y2) <= hitRadius) return ln;
    }
    return null;
  }

  const overlay = svg.append("rect")
    .attr("x", 0).attr("y", 0).attr("width", width).attr("height", height)
    .attr("fill", "transparent")
    .style("cursor", "pointer");

  overlay.on("pointermove", (event) => {
    if (event.pointerType === "touch") return;
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });
  overlay.on("pointerleave", () => hideTip());
  overlay.on("click", (event) => {
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });

  return svg.node();
}

html`<div class="ojs-plot-wrap">${dagSvg}</div>`

Example: political participation

• If we don’t include \(c\) in our model, we may over- or underestimate the (causal) relationship between \(X\) and \(Y\)
• Example: suppose we believe regional out-migration has a positive effect on support for populist parties
• Theory: in regions where a larger share of the population moves away, support for populist parties is higher
• But one reason people move is high unemployment, which itself affects support for populist parties
• If we don’t control for unemployment, we will overestimate the effect of out-migration on populist parties’ vote share

Room temperatures again

• Another example: think about the scenario where we want to explain households’ indoor temperature
• Last week we proposed a theory with the following prediction: households with high income are more likely to have higher room temperatures
• What’s a third factor that potentially affects both income and indoor temperature?
• Maybe education level?

Upward and downward biases

• Assume years of education has a positive effect on income but a negative effect on temperature

mutable biasModel = "bivariate"

html`<div class="sd-buttons-wrap">
  <div class="sd-buttons">
    ${Inputs.button("Bivariate model: income explains indoor temperature", {
      value: null,
      reduce: () => { mutable biasModel = "bivariate"; return null; }
    })}
    ${Inputs.button("Multivariate, controlling for education level", {
      value: null,
      reduce: () => { mutable biasModel = "multivariate"; return null; }
    })}
  </div>
</div>`

// v2 is only ever shown once the multivariate model is selected --
// column (2) stays visible (its header included) the whole time, but
// empty until then. Only the income *coefficient* row gets
// highlighted once both columns are visible (not its SE row below),
// so the jump in the estimate (0.015 -> 0.187) is easy to spot side
// by side without extra rows drawing the eye away from it.
biasRows = [
  { label: "Constant", v1: "18.363", v2: "17.039", isIncome: false },
  { label: "", v1: "(0.804)", v2: "(1.063)", isIncome: false },
  { label: "Income", v1: "0.015", v2: "0.187", isIncome: true },
  { label: "", v1: "(0.065)", v2: "(0.098)", isIncome: false },
  { label: "Years of education", v1: "", v2: "-0.208", isIncome: false },
  { label: "", v1: "", v2: "(0.111)", isIncome: false },
  { label: "Observations", v1: "100", v2: "100", isIncome: false },
  { label: "Adj. R²", v1: "0.00", v2: "0.02", isIncome: false },
  { label: "Root MSE", v1: "1.99", v2: "1.96", isIncome: false }
]

html`<div class="biasreg-table-wrap">
  <table class="biasreg-table">
    <thead><tr><th></th><th>(1)</th><th>(2)</th></tr></thead>
    <tbody>
      ${biasRows.map(r => {
        const showV2 = biasModel === "multivariate";
        const hl = showV2 && r.isIncome;
        const labelHtml = r.label === "Root MSE" ? html`<em>Root MSE</em>` : (r.label || " ");
        // A non-breaking space instead of "" for any empty cell -- a
        // literal empty string can render at a slightly shorter line
        // height than one with real text in some browsers, which was
        // making the (otherwise completely blank, in bivariate mode)
        // education-SE row visibly thinner than the rest and causing
        // the table to jump in height when switching models.
        const v1Text = r.v1 || " ";
        const v2Text = (showV2 ? r.v2 : "") || " ";
        return html`<tr>
          <td>${labelHtml}</td>
          <td class="${hl ? "biasreg-highlight" : ""}">${v1Text}</td>
          <td class="${hl ? "biasreg-highlight" : ""}">${v2Text}</td>
        </tr>`;
      })}
    </tbody>
  </table>
</div>`

• Not controlling for education level gives us a downward bias on the effect income has on households’ indoor temperature

Example of downward bias

• \(c\) has a positive effect on \(X\) and a negative effect on \(Y\), while \(X\) has a positive effect on \(Y\)

// Same running example as the "Upward and downward biases" slide:
// c = education level, X = income, Y = indoor temperature. The signs
// here (+, +, -) are exactly what produces that slide's downward bias
// when education is omitted -- reuses dagDistToSegment/dagWrapText
// from the earlier confounder-DAG slide rather than redefining them.
dag2Lines = [
  { key: "cx", x1: 100, y1: 460, x2: 100, y2: 65, sign: "+", signX: 55, signY: 267,
    text: "c affects X positively: e.g. higher education level leads on average to higher income" },
  { key: "xy", x1: 100, y1: 65, x2: 575, y2: 178, sign: "+", signX: 337, signY: 86,
    text: "X affects Y positively: e.g. higher income leads on average to higher indoor temperature" },
  { key: "cy", x1: 100, y1: 460, x2: 575, y2: 196, sign: "−", signX: 337, signY: 368,
    text: "c affects Y negatively: e.g. higher education level leads on average to lower indoor temperature" }
]

dag2Svg = {
  const width = 680, height = 520;
  const svg = d3.create("svg")
    .attr("viewBox", `0 0 ${width} ${height}`)
    .attr("width", width)
    .attr("height", height)
    .attr("class", "dag-svg")
    .style("touch-action", "none")
    .style("max-width", "100%")
    .style("display", "block")
    .style("margin", "0 auto");

  svg.append("defs").append("marker")
    .attr("id", "dag2-arrow")
    .attr("markerWidth", 14).attr("markerHeight", 14)
    .attr("refX", 11).attr("refY", 4.5)
    .attr("orient", "auto").attr("markerUnits", "strokeWidth")
    .append("path")
    .attr("d", "M0,0 L0,9 L13,4.5 z")
    .attr("fill", "currentColor");

  const lineEls = dag2Lines.map(ln => {
    const sel = svg.append("line")
      .attr("x1", ln.x1).attr("y1", ln.y1)
      .attr("x2", ln.x2).attr("y2", ln.y2)
      .attr("stroke", "#333").attr("stroke-width", 3)
      .attr("marker-end", "url(#dag2-arrow)");
    return { ...ln, sel };
  });

  svg.append("text").attr("x", 75).attr("y", 500).attr("font-size", 34).text("c");
  svg.append("text").attr("x", 75).attr("y", 55).attr("font-size", 34).text("X");
  svg.append("text").attr("x", 600).attr("y", 195).attr("font-size", 34).text("Y");

  const tipG = svg.append("g").attr("display", "none");
  const tipRect = tipG.append("rect")
    .attr("rx", 8).attr("fill", "white").attr("fill-opacity", 0.97)
    .attr("stroke", "#2563eb").attr("stroke-width", 2);
  const tipTextG = tipG.append("g");

  // Appended *after* the tooltip group, so the +/- signs always paint
  // on top of it (SVG stacks later siblings above earlier ones) --
  // otherwise a sign sitting near where the tooltip box happens to pop
  // up would get hidden behind it.
  const signsG = svg.append("g");
  dag2Lines.forEach(ln => {
    signsG.append("text")
      .attr("x", ln.signX).attr("y", ln.signY)
      .attr("text-anchor", "middle")
      .attr("font-size", 30).attr("font-weight", "bold")
      .attr("fill", ln.sign === "+" ? "#15803d" : "#b91c1c")
      .text(ln.sign);
  });

  const lineHeight = 22;
  const padX = 14, padY = 14;
  const maxCharsPerLine = 30;

  function showTip(mx, my, ln) {
    lineEls.forEach(o => o.sel
      .attr("stroke", o.key === ln.key ? "#2563eb" : "#333")
      .attr("stroke-width", o.key === ln.key ? 5 : 3));

    const lines = dagWrapText(ln.text, maxCharsPerLine);
    tipTextG.selectAll("*").remove();
    const boxW = Math.max(...lines.map(l => l.length)) * 9 + padX * 2;
    const boxH = lines.length * lineHeight + padY * 2 - 6;
    let bx = mx + 14, by = my - boxH - 14;
    if (bx + boxW > width - 2) bx = mx - boxW - 14;
    if (bx < 2) bx = 2;
    if (by < 2) by = my + 14;
    if (by + boxH > height - 2) by = height - boxH - 2;

    tipRect.attr("x", bx).attr("y", by).attr("width", boxW).attr("height", boxH);
    lines.forEach((line, i) => {
      tipTextG.append("text")
        .attr("x", bx + padX).attr("y", by + padY + 6 + i * lineHeight)
        .attr("font-size", 17).attr("fill", "#1e3a8a")
        .text(line);
    });
    tipG.attr("display", null);
  }

  function hideTip() {
    tipG.attr("display", "none");
    lineEls.forEach(o => o.sel.attr("stroke", "#333").attr("stroke-width", 3));
  }

  const hitRadius = 22;
  function findLine(mx, my) {
    for (const ln of lineEls) {
      if (dagDistToSegment(mx, my, ln.x1, ln.y1, ln.x2, ln.y2) <= hitRadius) return ln;
    }
    return null;
  }

  const overlay = svg.append("rect")
    .attr("x", 0).attr("y", 0).attr("width", width).attr("height", height)
    .attr("fill", "transparent")
    .style("cursor", "pointer");

  overlay.on("pointermove", (event) => {
    if (event.pointerType === "touch") return;
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });
  overlay.on("pointerleave", () => hideTip());
  overlay.on("click", (event) => {
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });

  return svg.node();
}

html`<div class="ojs-plot-wrap">${dag2Svg}</div>`

Other examples

• The combination of the relationship between \(c\) and \(X\), and \(c\) and \(Y\), is what determines the direction of the bias
• Upward bias: without controlling for \(c\), the estimated effect of \(X\) on \(Y\) will be more positive than the true effect
• Downward bias: without controlling for \(c\), the estimated effect of \(X\) on \(Y\) will be more negative than the true effect

// Plain Unicode superscript +/- for the header labels (avoids needing
// MathJax to reprocess dynamically-inserted content, which the LaTeX
// \xrightarrow{} version would have required). Each body cell combines
// its row's and column's sign into one explanation.
biasDirCells = [
  { row: 0, col: 0, label: "Upward bias", text: "Higher c leads on average to higher X and, at the same time, to higher Y" },
  { row: 0, col: 1, label: "Downward bias", text: "Higher c leads on average to lower X and, at the same time, to higher Y" },
  { row: 1, col: 0, label: "Downward bias", text: "Higher c leads on average to higher X and, at the same time, to lower Y" },
  { row: 1, col: 1, label: "Upward bias", text: "Higher c leads on average to lower X and, at the same time, to lower Y" }
]

// A single named cell that both builds and displays the table -- no
// separate trailing "biasDirTable" reference cell, which would
// re-display the very same (already-mounted) DOM node a second time
// and show up as a broken "HTMLDivElement {}" instead.
biasDirTable = {
  const outer = d3.create("div").attr("class", "bias-dir-outer");
  const table = outer.append("table").attr("class", "bias-dir-table");
  const noteDiv = outer.append("div").attr("class", "bias-dir-note");

  const colHeaders = ["c →⁺ X", "c →⁻ X"];
  const rowHeaders = ["c →⁺ Y", "c →⁻ Y"];

  const theadRow = table.append("thead").append("tr");
  theadRow.append("th");
  colHeaders.forEach(h => theadRow.append("th").text(h));

  const tbody = table.append("tbody");
  rowHeaders.forEach((rh, row) => {
    const tr = tbody.append("tr");
    tr.append("th").text(rh);
    [0, 1].forEach(col => {
      const cellDef = biasDirCells.find(c => c.row === row && c.col === col);
      tr.append("td")
        .attr("class", "bias-dir-cell")
        .text(cellDef.label)
        .on("pointerenter", (event) => {
          if (event.pointerType === "touch") return;
          noteDiv.text(cellDef.text);
        })
        .on("pointerleave", (event) => {
          if (event.pointerType === "touch") return;
          noteDiv.text("");
        })
        .on("click", () => {
          noteDiv.text(cellDef.text);
        });
    });
  });

  return outer.node();
}

Interpreting the slope coefficients

• In the bivariate case: we interpret \(b\) as the expected increase in the dependent variable when the independent variable increases by one unit
• In the multiple regression case, we add “holding everything else constant”
• Consider the second column in the table on the previous slide: an increase of 1000 SEK in monthly income is expected to increase room temperature by 0.187 degrees Celsius, holding everything else constant
• “Holding everything else constant” here means the effect income has on room temperature while holding all other variables constant

Interpreting the size of the coefficient

• How do we know if the effect we’re estimating is large or small?
• Last week we distinguished between good fit (high \(R^2\) and/or low Root-MSE) and a strong relationship (large positive/negative \(b\))
• However, large positive or negative estimates of \(b\) don’t necessarily tell us whether the relationship is “meaningful”
• Example: suppose we estimate the relationship between local unemployment and support for populist parties
• In our regression model we estimate \(b =\) 0.2: seems like a small effect, right?

Large/small change in \(Y\) and/or \(X\)

• Suppose local unemployment varies from 2 to 20%
• At the same time, the local vote share for the populist party varies between 3 and 6%
• A one-unit increase in unemployment is therefore a small change relative to the range of the independent variable
• Its effect, 0.2, is fairly large compared to the total vote share for populist parties

		Change in X
		Small	Large
Change in Y	Small	Moderate	Weak
	Large	Strong	Moderate

Range of \(X\) and \(Y\)

• So what counts as a large change in \(X\) and \(Y\)?
• One way is to look at the spread of these variables
• Consider once again the room-temperature example:

	(1)	(2)
Intercept	18.363	17.039
	(0.804)	(1.063)
Income	0.015	0.187
	(0.065)	(0.098)
Years of education		-0.208
		(0.111)
Observations	100	100
Adj. \(R^2\)	0.00	0.02
Root MSE	1.99	1.96

How much is one unit?

• Suppose we have the following table of descriptive statistics:

Variable	Mean	s.d.	Min	Max
Temperature	18.71	2.01	14.30	22.80
Income	22.36	5.82	9.78	36.98
Years of education	12.06	5.13	0.00	26

• The estimated slope coefficient for income is 0.187, and it’s statistically significant at the 90% confidence level (why?)
• The effect of income on temperature is 0.187, which is small relative to the range of the dependent variable, which is 22.80 \(-\) 14.3 \(=\) 8.5
• The range of income is 36.98 \(-\) 9.78 \(=\) 27.2, so a one (1) unit increase in income is fairly small compared to its range
• A small change in \(X\) gives a small change in \(Y\): Moderate effect

Model fit with multiple variables

• Using \(R^2\) to assess fit is problematic in a multiple regression model
• We can always increase, or at least not decrease, \(R^2\) by including more variables in our model
• We prefer as small a model as possible that explains as much as possible
• Adjusted \(R\)-squared “penalizes” the inclusion of independent variables
• The interpretation of Root-MSE remains the same

Bad controls

• Earlier we said we must control for confounders
• Consider this example instead:

// Mediator DAG (X -> m -> Y, plus a direct X -> Y), same running
// example as the following slides: X = education level,
// m = sector, Y = wage. Reuses dagDistToSegment/dagWrapText from the
// confounder-DAG slide earlier in this deck.
dag3Lines = [
  { key: "xm", x1: 100, y1: 65, x2: 100, y2: 460,
    text: "X affects m: e.g. higher education level leads on average to employment in the high-skill sector" },
  { key: "my", x1: 100, y1: 460, x2: 575, y2: 196,
    text: "m affects Y: e.g. employment in the high-skill sector leads on average to a higher wage" },
  { key: "xy", x1: 100, y1: 65, x2: 575, y2: 178,
    text: "X affects Y directly: the effect of education on wages beyond sector choice is close to zero, since there's hardly any wage variation within each sector" }
]

dag3Svg = {
  const width = 680, height = 520;
  const svg = d3.create("svg")
    .attr("viewBox", `0 0 ${width} ${height}`)
    .attr("width", width)
    .attr("height", height)
    .attr("class", "dag-svg")
    .style("touch-action", "none")
    .style("max-width", "100%")
    .style("display", "block")
    .style("margin", "0 auto");

  svg.append("defs").append("marker")
    .attr("id", "dag3-arrow")
    .attr("markerWidth", 14).attr("markerHeight", 14)
    .attr("refX", 11).attr("refY", 4.5)
    .attr("orient", "auto").attr("markerUnits", "strokeWidth")
    .append("path")
    .attr("d", "M0,0 L0,9 L13,4.5 z")
    .attr("fill", "currentColor");

  const lineEls = dag3Lines.map(ln => {
    const sel = svg.append("line")
      .attr("x1", ln.x1).attr("y1", ln.y1)
      .attr("x2", ln.x2).attr("y2", ln.y2)
      .attr("stroke", "#333").attr("stroke-width", 3)
      .attr("marker-end", "url(#dag3-arrow)");
    return { ...ln, sel };
  });

  svg.append("text").attr("x", 75).attr("y", 500).attr("font-size", 34).text("m");
  svg.append("text").attr("x", 75).attr("y", 55).attr("font-size", 34).text("X");
  svg.append("text").attr("x", 600).attr("y", 195).attr("font-size", 34).text("Y");

  const tipG = svg.append("g").attr("display", "none");
  const tipRect = tipG.append("rect")
    .attr("rx", 8).attr("fill", "white").attr("fill-opacity", 0.97)
    .attr("stroke", "#2563eb").attr("stroke-width", 2);
  const tipTextG = tipG.append("g");

  const lineHeight = 22;
  const padX = 14, padY = 14;
  const maxCharsPerLine = 30;

  function showTip(mx, my, ln) {
    lineEls.forEach(o => o.sel
      .attr("stroke", o.key === ln.key ? "#2563eb" : "#333")
      .attr("stroke-width", o.key === ln.key ? 5 : 3));

    const lines = dagWrapText(ln.text, maxCharsPerLine);
    tipTextG.selectAll("*").remove();
    const boxW = Math.max(...lines.map(l => l.length)) * 9 + padX * 2;
    const boxH = lines.length * lineHeight + padY * 2 - 6;
    let bx = mx + 14, by = my - boxH - 14;
    if (bx + boxW > width - 2) bx = mx - boxW - 14;
    if (bx < 2) bx = 2;
    if (by < 2) by = my + 14;
    if (by + boxH > height - 2) by = height - boxH - 2;

    tipRect.attr("x", bx).attr("y", by).attr("width", boxW).attr("height", boxH);
    lines.forEach((line, i) => {
      tipTextG.append("text")
        .attr("x", bx + padX).attr("y", by + padY + 6 + i * lineHeight)
        .attr("font-size", 17).attr("fill", "#1e3a8a")
        .text(line);
    });
    tipG.attr("display", null);
  }

  function hideTip() {
    tipG.attr("display", "none");
    lineEls.forEach(o => o.sel.attr("stroke", "#333").attr("stroke-width", 3));
  }

  const hitRadius = 22;
  function findLine(mx, my) {
    for (const ln of lineEls) {
      if (dagDistToSegment(mx, my, ln.x1, ln.y1, ln.x2, ln.y2) <= hitRadius) return ln;
    }
    return null;
  }

  const overlay = svg.append("rect")
    .attr("x", 0).attr("y", 0).attr("width", width).attr("height", height)
    .attr("fill", "transparent")
    .style("cursor", "pointer");

  overlay.on("pointermove", (event) => {
    if (event.pointerType === "touch") return;
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });
  overlay.on("pointerleave", () => hideTip());
  overlay.on("click", (event) => {
    const [mx, my] = d3.pointer(event);
    const ln = findLine(mx, my);
    if (ln) showTip(mx, my, ln); else hideTip();
  });

  return svg.node();
}

html`<div class="ojs-plot-wrap">${dag3Svg}</div>`

• We now have a mediator: \(m\)

Bad controls: an example

• \(X\) has a direct effect on \(Y\) but also an indirect one through its effect on \(m\)
• If we include \(m\) in the model, we “absorb” part of the (total) effect \(X\) has on \(Y\)
• Example: suppose there are two separate labor market sectors: a high-skill and a low-skill sector
• The high-skill sector pays a much higher wage than the low-skill sector
• Suppose it’s nearly impossible to get hired in the high-skill sector without an advanced degree, and that there’s practically no wage variation within each sector

Friends don’t let friends include bad controls

• Suppose we want to examine the effect of education level on wages
• If we control for sector, we will estimate no or a very small effect of education level on wages
• Remember the interpretation in a multiple regression: the effect of education on wages while holding sector constant
• Within each sector, almost all workers have the same (or no) degree and earn the same wage
• Sector is, in this case, a mediator

Controlling: what to do and not do

• Think of the causal link as a stream
• If the confounder is upstream, i.e. it was determined before the variable you’re interested in: include it
• If the confounder is downstream, i.e. it was determined after the variable you’re interested in: don’t include it
• What do you do if the variable you’re interested in and the confounder affect each other?
• Adding controls to identify the causal link doesn’t always work: we may need to look for random variation / run experiments instead

Example exam question

• Understanding regression analysis lets you interpret results in regression tables
• Let’s go back to the table we saw earlier

// Shared by this slide and "Interpreting the results: no controls"
// right after it -- same data, same interactivity, built via a
// factory function (buildTentaTable) rather than a single value, since
// the two slides each need their *own* table instance (a DOM node can
// only live in one place at a time).
tentaRows = [
  { label: "The model's intercept",
    v1: { val: "18.363", key: "const1", desc: "Intercept/constant for the bivariate model" },
    v2: { val: "17.039", key: "const2", desc: "Intercept/constant for the multivariate model" } },
  { label: "",
    v1: { val: "(0.804)", key: "const1se", desc: "Standard error for the intercept/constant in the bivariate model" },
    v2: { val: "(1.063)", key: "const2se", desc: "Standard error for the intercept/constant in the multivariate model" } },
  { label: "Monthly income (in 1000 SEK)",
    v1: { val: "0.015", key: "inc1", desc: "Slope coefficient for monthly income (in 1000 SEK) in the bivariate model" },
    v2: { val: "0.187", key: "inc2", desc: "Slope coefficient for monthly income (in 1000 SEK) in the multivariate model" } },
  { label: "",
    v1: { val: "(0.065)", key: "inc1se", desc: "Standard error for the slope coefficient for monthly income in the bivariate model" },
    v2: { val: "(0.098)", key: "inc2se", desc: "Standard error for the slope coefficient for monthly income in the multivariate model" } },
  { label: "Years of education",
    v1: { val: "", key: null, desc: null },
    v2: { val: "-0.208", key: "edu2", desc: "Slope coefficient for years of education in the multivariate model" } },
  { label: "",
    v1: { val: "", key: null, desc: null },
    v2: { val: "(0.111)", key: "edu2se", desc: "Standard error for the slope coefficient for years of education in the multivariate model" } },
  { label: "Observations",
    v1: { val: "100", key: "obs1", desc: "Number of observations in the bivariate model" },
    v2: { val: "100", key: "obs2", desc: "Number of observations in the multivariate model" } },
  { label: "Adj. R²",
    v1: { val: "0.00", key: "r2_1", desc: "Adjusted R² for the bivariate model" },
    v2: { val: "0.02", key: "r2_2", desc: "Adjusted R² for the multivariate model" } },
  { label: "Root MSE", italic: true,
    v1: { val: "1.99", key: "rmse1", desc: "Root-MSE for the bivariate model" },
    v2: { val: "1.96", key: "rmse2", desc: "Root-MSE for the multivariate model" } }
]

// highlightKeys marks specific cells (independent of, and layered on
// top of, each cell's own hover/click tooltip) -- used by the
// step-by-step slide right after this one. Hover/click pops up a
// small floating box positioned above (or below, near the top/bottom
// edge) the cell itself.
function buildTentaTable({ highlightKeys = [] } = {}) {
  const outer = d3.create("div").attr("class", "tenta-table-outer");
  const tableWrap = outer.append("div").attr("class", "tenta-table-wrap");
  const table = tableWrap.append("table").attr("class", "tenta-table");
  const tip = tableWrap.append("div").attr("class", "tenta-tip").style("display", "none");

  function showTip(cellNode, desc) {
    tip.text(desc).style("display", "block");
    const wrapRect = tableWrap.node().getBoundingClientRect();
    const cellRect = cellNode.getBoundingClientRect();
    const tipNode = tip.node();
    let left = (cellRect.left - wrapRect.left) + cellRect.width / 2 - tipNode.offsetWidth / 2;
    let top = (cellRect.top - wrapRect.top) - tipNode.offsetHeight - 8;
    if (left < 0) left = 0;
    if (left + tipNode.offsetWidth > wrapRect.width) left = wrapRect.width - tipNode.offsetWidth;
    if (top < 0) top = (cellRect.bottom - wrapRect.top) + 8;
    tip.style("left", `${left}px`).style("top", `${top}px`);
  }
  function hideTip() {
    tip.style("display", "none");
  }

  const theadRow = table.append("thead").append("tr");
  theadRow.append("th");
  theadRow.append("th").text("(1)");
  theadRow.append("th").text("(2)");

  const tbody = table.append("tbody");
  tentaRows.forEach(r => {
    const tr = tbody.append("tr");
    tr.append("td").attr("class", "tenta-label").text(r.label);
    [r.v1, r.v2].forEach(cellData => {
      const td = tr.append("td").text(cellData.val);
      if (r.italic) td.style("font-style", "italic");
      if (cellData.key && highlightKeys.includes(cellData.key)) {
        td.classed("tenta-highlight", true);
      }
      if (cellData.desc) {
        td.classed("tenta-interactive", true)
          .on("pointerenter", function (event) {
            if (event.pointerType === "touch") return;
            showTip(this, cellData.desc);
          })
          .on("pointerleave", (event) => {
            if (event.pointerType === "touch") return;
            hideTip();
          })
          .on("click", function () {
            showTip(this, cellData.desc);
          });
      }
    });
  });

  return outer.node();
}

html`<div class="ojs-plot-wrap">${buildTentaTable()}</div>`

Interpreting the results: no controls

mutable tentaStep = 1

tentaStepBullets = [
  "Understand what <strong>unit(s)</strong> the dependent variable and independent variables are measured in: Y = temperature in degrees Celsius; X₁ = monthly income in 1000 SEK; X₂ = education, in years",
  "Interpret the slope coefficient for income in column 1: an increase of 1000 SEK in monthly income is expected to increase room temperature by 0.015 degrees C",
  "Is the relationship statistically significant? Use the estimate, its standard error, and the appropriate <em>t</em>-value",
  "Start with 95%; if it's significant at that confidence level, then test 99% (unless stated otherwise)",
  "Summarize the relationship: There is a <strong>positive</strong> but <strong>not statistically significant</strong> relationship between income and room temperature. (Or, since it's not significant: there is no relationship between income and room temperature.)"
]

// Which table cells (by key, see tentaRows above) light up at each
// step -- index 0 = step 1, etc.
tentaStepHighlights = [
  [],
  ["inc1"],
  ["inc1", "inc1se"],
  ["inc1", "inc1se"],
  []
]

html`<div class="sd-buttons-wrap"><div class="sd-buttons">
  ${Inputs.button("← Previous step", {
    value: null,
    reduce: () => { mutable tentaStep = Math.max(1, tentaStep - 1); return null; }
  })}
  ${Inputs.button("Next step →", {
    value: null,
    reduce: () => { mutable tentaStep = Math.min(tentaStepBullets.length, tentaStep + 1); return null; }
  })}
</div></div>`

// All steps are always visible (bullets exist the whole time, only
// *which one is highlighted* changes) -- rebuilt each step change,
// cheap with only 5 items.
tentaStepsList = {
  const wrap = d3.create("div").attr("class", "tenta-step-list-wrap");
  const ol = wrap.append("ol").attr("class", "tenta-steps-list");
  tentaStepBullets.forEach((htmlStr, i) => {
    ol.append("li")
      .attr("class", i + 1 === tentaStep ? "tenta-step-active" : "")
      .html(htmlStr);
  });
  return wrap.node();
}

html`<div class="ojs-plot-wrap">${buildTentaTable({ highlightKeys: tentaStepHighlights[tentaStep - 1] })}</div>`

// Appears from step 4 onward (min-height reserved below so the column
// doesn't jump when it does).
tentaCiEq = {
  const wrap = d3.create("div").attr("class", "tenta-ci-eq");
  if (tentaStep >= 4) {
    const lines = [
      "b ± t × se(b)",
      "= 0.015 ± 1.96 × 0.065",
      "= 0.015 ± 0.127",
      "= [−0.112, 0.142]  (covers 0)"
    ];
    lines.forEach(l => wrap.append("div").attr("class", "tenta-ci-eq-line").text(l));
  }
  return wrap.node();
}

Interpreting the results: with control variables

• Now let’s look at the second column

mutable tentaStep2 = 1

tentaStepBullets2 = [
  "Interpret the slope coefficient for income in column 2: an increase of 1000 SEK in monthly income is expected to increase room temperature by 0.187 degrees C, holding everything else constant (with education level held constant)",
  "Is the relationship statistically significant? Use the estimate, its standard error, and the appropriate <em>t</em>-value",
  "A 95% CI is [−0.005, 0.379] and covers 0, so the relationship is not statistically significant at the 95% confidence level",
  "If the question states that you should test down to, e.g., 90%, then calculate a 90% CI if the 95% one doesn't cover 0",
  "A 90% CI is [0.025, 0.349] and doesn't cover 0, so the relationship is statistically significant at the 90% confidence level",
  "Summarize the relationship: There is a <strong>positive</strong> relationship between income and room temperature, which is <strong>statistically significant</strong> at the 90% confidence level",
  "Do the same for the control variables, if you're asked to (their interpretation is usually less relevant)",
  "If asked about the model's fit, you can comment on R², Adj.-R², or Root-MSE"
]

tentaStepHighlights2 = [
  ["inc2"],
  ["inc2", "inc2se"],
  ["inc2", "inc2se"],
  ["inc2", "inc2se"],
  ["inc2", "inc2se"],
  [],
  ["edu2", "edu2se"],
  ["r2_2", "rmse2"]
]

html`<div class="sd-buttons-wrap"><div class="sd-buttons">
  ${Inputs.button("← Previous step", {
    value: null,
    reduce: () => { mutable tentaStep2 = Math.max(1, tentaStep2 - 1); return null; }
  })}
  ${Inputs.button("Next step →", {
    value: null,
    reduce: () => { mutable tentaStep2 = Math.min(tentaStepBullets2.length, tentaStep2 + 1); return null; }
  })}
</div></div>`

tentaStepsList2 = {
  const wrap = d3.create("div").attr("class", "tenta-step-list-wrap");
  const ol = wrap.append("ol").attr("class", "tenta-steps-list");
  tentaStepBullets2.forEach((htmlStr, i) => {
    ol.append("li")
      .attr("class", i + 1 === tentaStep2 ? "tenta-step-active" : "")
      .html(htmlStr);
  });
  return wrap.node();
}

html`<div class="ojs-plot-wrap">${buildTentaTable({ highlightKeys: tentaStepHighlights2[tentaStep2 - 1] })}</div>`

// Steps 1-2: nothing shown yet. Steps 3-4: the 95% calculation (step 4
// is the "if the question asks, go to 90%" instruction, which still
// refers back to this same 95% result). Steps 5-6: swapped for the 90%
// calculation only -- once the narrative has moved on to 90%, the 95%
// one no longer needs to stay visible. Steps 7-8 (control variables,
// model fit) aren't about the CI at all, so the box stays empty again.
tentaCiEq2 = {
  const wrap = d3.create("div").attr("class", "tenta-ci-eq");
  if (tentaStep2 === 3 || tentaStep2 === 4) {
    const lines95 = [
      "b ± t × se(b)",
      "= 0.187 ± 1.96 × 0.098",
      "= 0.187 ± 0.192",
      "= [−0.005, 0.379]  (covers 0)"
    ];
    lines95.forEach(l => wrap.append("div").attr("class", "tenta-ci-eq-line").text(l));
  } else if (tentaStep2 === 5 || tentaStep2 === 6) {
    const lines90 = [
      "b ± t × se(b)",
      "= 0.187 ± 1.65 × 0.098",
      "= 0.187 ± 0.162",
      "= [0.025, 0.349]  (doesn't cover 0)"
    ];
    lines90.forEach(l => wrap.append("div").attr("class", "tenta-ci-eq-line").text(l));
  }
  return wrap.node();
}