Key Points

Introduction to Data Processing


  • Processing turns collected data into analysis-ready data: inspect, clean, transform, combine, derive
  • Keep raw data unchanged and write results somewhere else
  • Record every step: a notebook or script, the QGIS Processing History or a model, and a short log
  • Check after every step, with counts, totals and a quick map
  • Use QGIS for quick, visual work and Python (or a QGIS model) for work you will repeat

Cleaning and Preparing Data in Python


  • Inspect before changing anything: shape, types, minimum and maximum reveal most problems
  • Read identifiers as text, convert numbers stored as text, and turn special codes into missing values
  • ACS estimates have margins of error; the coefficient of variation shows which ones are too uncertain to rank
  • Decide what makes two records the same before removing duplicates
  • Define a CRS to say what coordinates are; reproject to change them
  • Check every step with a count, a join or a map, and write your decisions down

Cleaning and Preparing Data in QGIS


  • Inspect before changing anything: sorting a column quickly shows text that should be numbers and special codes
  • Refactor Fields renames fields, sets their types and fills them with expressions in one step, without changing the original
  • to_int(), if(... < 0, NULL, ...) and nullif(x, 0) handle text numbers, codes and division by zero
  • ACS estimates have margins of error; the coefficient of variation shows which are too uncertain to rank
  • Create points from coordinates with the CRS they are in; decide what makes two records the same before deleting
  • Check validity, remove unwanted features and dissolve split ones, and confirm each step with a count

Geoprocessing in Python


  • Project every layer to a CRS in meters (EPSG:26916 for Indiana) before buffering, measuring or overlaying
  • Buffer draws zones, dissolve merges them, clip and intersect cut shapes, difference keeps what lies outside, and a nearest join finds the closest feature and its distance
  • The spatial predicate (intersects, within, centroid within) changes which features are selected; report it
  • Keep the original area before cutting, so areal weighting can split counts by area share
  • Sum counts, never rates or medians; areal weighting assumes an even spread and straight lines are not travel

Geoprocessing in QGIS


  • Reproject every layer to a CRS in meters (EPSG:26916 for Indiana) before buffering, measuring or overlaying; on-the-fly display does not change the data
  • Buffer draws zones, Dissolve merges them, Clip and Intersection cut shapes, Difference keeps what lies outside, and Join attributes by nearest finds the closest feature and its distance
  • The predicate in Select by Location (intersect, are within, centroid within) changes the answer; report it
  • Store the original area before cutting, then use area($geometry) / "tract_m2" * count for areal weighting
  • Sum counts, never rates or medians; processing results are temporary until you make them permanent

Raster Processing in Python


  • Map algebra works cell by cell on rasters that share CRS, cell size and extent; resample categories with nearest neighbor
  • Comparisons make Boolean rasters, lookup tables reclassify codes, and & combines conditions
  • Zonal statistics summarize cells inside polygons: means for measurements, class counts for categories
  • Cell counts times cell area give areas, in a CRS measured in meters
  • Dasymetric estimation uses land cover to place people; compare it with areal weighting

Raster Processing in QGIS


  • Rasters combined cell by cell must share CRS, cell size and extent; resample categories with nearest neighbor
  • The Raster Calculator makes comparisons and combines conditions (AND, OR); results are 1 and 0
  • Reclassify by table groups codes; set the range boundaries to min <= value <= max for whole-number codes
  • Zonal statistics summarize measurements per polygon; Zonal histogram counts categories per polygon
  • Check results with the unique values report: cells times 900 m² gives areas

Automating Repeated Steps in Python


  • Automate work you will repeat; record work you will not
  • A function packages steps; parameters are the parts that change, defaults the usual values
  • Test on a known answer, check inputs, and report counts so every run leaves evidence
  • Loops (or groupby) run the same workflow for many inputs and collect the results in one table
  • Combine units by adding counts first, then computing rates
  • Catch errors in long loops, but record them and check the totals at the end

Automating Repeated Steps in QGIS


  • Automate work you will repeat; record work you will not (Processing > History)
  • A model packages several tools; its inputs are the parts that change
  • Test a model on a known answer before using it on new questions
  • Run as Batch Process repeats a tool or model with different settings, one row per run
  • Iterate over this layer runs a tool once per feature
  • Make common mistakes impossible inside the model, for example by reprojecting every input