<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>hexagons_world-truth.txt</title>
  <script>
    (() => {
      let storedTheme = null;
      try {
        storedTheme = localStorage.getItem("theme");
      } catch (error) {}

      if (storedTheme === "dark") {
        document.documentElement.dataset.theme = "dark";
      }
    })();
  </script>
  <link rel="stylesheet" href="/%25latex/styles.css">
</head>
<body class="document-page">
  <button class="theme-toggle" type="button" aria-label="Switch to dark mode" title="Dark mode">
    <svg class="theme-icon theme-icon-moon" viewBox="0 0 24 24" aria-hidden="true">
      <path d="M20.4 14.5A8.2 8.2 0 0 1 9.5 3.6a8.7 8.7 0 1 0 10.9 10.9Z"></path>
    </svg>
    <svg class="theme-icon theme-icon-sun" viewBox="0 0 24 24" aria-hidden="true">
      <circle cx="12" cy="12" r="4"></circle>
      <path d="M12 2v2M12 20v2M4.9 4.9l1.4 1.4M17.7 17.7l1.4 1.4M2 12h2M20 12h2M4.9 19.1l1.4-1.4M17.7 6.3l1.4-1.4"></path>
    </svg>
  </button>
  <script>
    (() => {
      const button = document.querySelector(".theme-toggle");

      const setTheme = (theme) => {
        const isDark = theme === "dark";
        if (isDark) {
          document.documentElement.dataset.theme = "dark";
        } else {
          document.documentElement.removeAttribute("data-theme");
        }

        button.setAttribute("aria-label", isDark ? "Switch to light mode" : "Switch to dark mode");
        button.setAttribute("aria-pressed", String(isDark));
        button.title = isDark ? "Light mode" : "Dark mode";

        try {
          localStorage.setItem("theme", isDark ? "dark" : "light");
        } catch (error) {}
      };

      setTheme(document.documentElement.dataset.theme === "dark" ? "dark" : "light");
      button.addEventListener("click", () => {
        setTheme(document.documentElement.dataset.theme === "dark" ? "light" : "dark");
      });
    })();
  </script>
  <header class="site-header">
    <a class="back-link" href="/outline/hexagons_world/">Back to directory</a>
    <h1>hexagons_world-truth.txt</h1>
    <p class="path-label">outline/hexagons_world/hexagons_world-truth.txt</p>
  </header>
  <main class="document-shell">
    <pre class="document-text">## hexagons.world

### Truth

Data about the world sits in thousands of separate places. Satellite archives, census bureaus, NGO dashboards, and corporate servers each run on their own schedule, format, and access rules. What is free and global is often coarse; what is detailed is often patchy, delayed, or locked behind expertise that most organisations do not have.

Even when data is technically open, it frequently arrives in PDFs, obscure formats, or portals that assume fluency in SQL and geospatial file types. Organisations generate rich information constantly — transactions, sensor readings, observations, forms — but much of it is never gathered in one place. The people who could act on it rarely have the tools to clean, combine, and explore it.

The dominant analysis platform is Esri ArcGIS, priced so that small organisations routinely exclude it. The open-source QGIS, PostGIS, and GeoServer stack is free but requires technical assembly. Google Earth Engine offers extraordinary compute and data but runs as an unauditable black box that cannot be self-hosted or offered to clients as part of a transparent workflow.

#### Change

	useful
		active citizens
			notice something near them and adjust it
			find others who care and act together
			learn what works in practice, not just in theory
			quietly change how a street, building, or service feels

		not enough on its own
			protests
				show that something is wrong
				create visibility
				but rarely give people clear next steps where they live
				often fade once the moment passes

		the gap
			most effort is either:
				brief and visible (marches, campaigns)
				or slow and hidden (local work, small groups)
			without a shared record, it’s hard to see how they connect
			people don’t know:
				what has already been tried
				where things are slowly improving
				where help is still missing


#### Where people look for data now

	There is no single home for statistics.
	People search across many portals, each partial:
		- UNdata, World Bank, IMF, WHO, ILO, OECD.
		- National open data portals where they exist.
		- NGO dashboards (Global Forest Watch, Our World in Data).
		- Academic repositories and scattered PDF reports.
	These sources are valuable, but fragmented.
	They differ in frequency, quality, and coverage, and they rarely speak to each other.
	What’s missing is a simple, coherent map of what the world already knows.



##### What’s openly available worldwide

	Level 0 – Satellites and sensors
		Global, free, and always on.
		→ Night-time lights → proxy for economic activity.
		→ Land cover, deforestation, crop yields.
		→ Air pollution and climate indicators.
		Strength → consistent everywhere.
		Weakness → coarse, not personal.

	Level 1 – Core demographics
		Censuses, births and deaths, UN estimates.
		Strength → strong reach.
		Weakness → often out of date in fragile states.

	Level 2 – Economy and services
		GDP, inflation, trade.
		Employment, schooling, literacy.
		Health indicators.
		Strength → builds the outer frame.
		Weakness → not fine-grained.

	Level 3 – Social and wellbeing
		Surveys (Gallup, World Values, Afrobarometer, Eurobarometer).
		National wellbeing dashboards.
		NGO/university reports on loneliness, nutrition, homelessness.
		Strength → adds human texture.
		Weakness → scattered, inconsistent.

	Level 4 – Behavioural and real-time
		Google Trends, Wikipedia views, YouTube trending.
		Social media corpora, mobility data.
		Strength → fast, granular.
		Weakness → biased toward connected users.

	Level 5 – Composite indices
		HDI, Freedom House, Corruption Perceptions, Happiness Report.
		Strength → simple comparisons.
		Weakness → hides nuance, depends on earlier levels.


##### How coverage differs by country

	Developed countries
		Regular censuses, digitised records, open portals.
		Reliable, fine-grained, slow to publish.

	Emerging economies
		Censuses exist but surveys patchy.
		Records fragmented, NGOs fill gaps.
		Reliability mixed.

	Developing countries
		Censuses rare, surveys donor-funded.
		UN/NGO issue-specific data common.
		Satellites fill environmental/demographic gaps.
		Reliability patchy, often delayed.


##### Usability

	Even when data is “open,” it is often locked away in PDFs, obscure formats, or portals requiring expertise.
	Communities, councils, or students rarely have the tools to combine these sources.
	Researchers spend more time cleaning data than learning from it.
	Open does not always mean usable.


##### Ways to fill the gaps

	Interpolation → bridge between survey years.
	Extrapolation → extend trends.
	Regression proxies → e.g. lights for GDP.
	Bayesian models → combine weak sources with explicit uncertainty.
	Small-area estimation → apply national data to local hexagons.
	Composite indices → blend into single scores.

	these have stengths and weaknesses, which can be made transparent, as long as you know the reliability of the data types

#### Analysing data

Organisations everywhere create, deal with, and need to analyse data.

	create
		They create it constantly, often without calling it that.
		Every transaction, visit, sensor reading, form, and observation is a signal.
		Many industries already treat this as an asset.
		But there are just as many — small councils, community groups, family services, local retailers — that generate rich information and never gather it in one place.
		The data exists; it just hasn't been collected.

	analyse
		Public datasets are only useful if people can actually work with them.
		The current ecosystem of tools assumes its user is a geospatial analyst or a statistician.
		Someone who speaks SQL, knows their way around a shapefile, and can clean a messy CSV without flinching.
		That is not most people in most organisations.
		A charity coordinator, a council planner, a student researcher, a small-business owner — they have questions that data could answer, but not the months it takes to learn the tooling.

	organisations also deal with
		messy, unstructured, confusing data.
		It arrives in spreadsheets with inconsistent date formats, in PDFs that need to be scraped, in legacy databases no one fully understands, in handwritten survey notes, in third-party exports with columns that change every quarter.
		Before any analysis happens, someone has to reconcile four different versions of the same thing.
		This invisible cleaning work consumes most of the time that was supposed to be spent on insight.

	uploading your own datasets
		When data is sensitive — patient records, donor lists, proprietary surveys, infrastructure details — it cannot simply be uploaded to a black-box cloud service in another legal jurisdiction.
		Organisations need to know where their data lives.
		They need it to integrate with existing databases and data warehouses, not to sit in yet another silo.
		For some sectors — healthcare, utilities, government, mining — this is non-negotiable.
		The tooling must run where they choose: on their premises, in their cloud account, in a sovereign data centre they can audit.

	maps
		Tables hide patterns that maps make obvious.
		A column of postcodes does not tell you which neighbourhoods cluster together or which services are close enough to share a catchment.
		A time-series of incidents does not show you the street corner where they concentrate.
		Spatial display turns "what happened" into "where it happened," and that shift matters because action is local.
		People naturally think in terms of here and there, near and far, this area and the one next to it.
		A map matches the mental model; a spreadsheet fights it.



##### Who's analysing data?

	*non-government organisations*
		charities
			need to show donors where money goes and where it matters most
			want to map need, measure impact, and decide where to put limited resources
			showing their work on a map makes the case visible in a way tables cannot

		aid organisations
			operate across regions with patchy official data
			build their own pictures of ground truth to guide strategy
			must demonstrate outcomes to maintain funding

	*universities*
		produce research that must be published, cited, and sometimes turned into policy advice
		strategic plans increasingly demand evidence of regional engagement and real-world impact

	*researchers*
		think-tanks and independent analysts study trends and publish findings
		need reliable, combinable sources they can reference with confidence

	*government*
		local governments plan services, allocate budgets, and report to citizens
		state and federal agencies hold enormous datasets but often struggle to combine them across departments or present them clearly

	*corporate*
		mining
			model terrain, environmental risk, and regulatory boundaries
		insurance
			analyse weather patterns, flood zones, and claims geography
			build predictive models across regions
		healthcare
			map disease burden, service access, and patient flow
			often handle sensitive workloads that must stay in controlled environments
		agriculture
			track yield, soil health, water availability, and climate exposure
		retail
			study foot traffic, catchment demographics, and location performance
		telecommunications
			plan coverage, capacity, and infrastructure rollout
		many other industries
			any organisation that operates in more than one place eventually needs to see those places compared



#### Which tools are they using?

Most tools are extremely expensive and not suitable for people without a geospatial background.


##### Esri ArcGIS

		ArcGIS is the incumbent. It is also extremely expensive.

		ArcGIS Pro
			desktop suite
			powerful and comprehensive
			priced so that small organisations routinely exclude it from their budgets

		ArcGIS Enterprise
			your own server farm
			ludicrously expensive once licensing, maintenance, consultant fees, and upgrade cycles are counted

		ArcGIS Online
			cloud path with lower upfront cost
			but deep vendor lock-in: your data, styling, and workflows become difficult to extract and move elsewhere
			the licensing changes frequently, which means organisations built around one version find themselves renegotiating terms they did not anticipate

		I call this software sales approach the 'barrel' strategy, and people are generally pretty frustrated by it.


##### QGIS + PostGIS + GeoServer
		open-source stack
		completely free
		and pretty good
		QGIS handles the desktop work, PostGIS stores spatial data with proper queries, GeoServer serves tiles and layers
		it requires more assembly than a commercial suite, but it belongs to the user and the file formats are open
		for organisations with technical staff, this is a genuine alternative


##### Google Earth Engine

		joke: run by Mum and Pop Shop, Google.

		pulls in extraordinary data from many different sources
			NASA satellite archives, elevation models, climate records, land-cover sets

		runs on their global network
			essentially, Google has a lot of compute available
			can split a workload across thousands of idle machines and finish in minutes what would take days elsewhere

		they have also made the use of Google Earth Engine free for not-for-profits
			which is genuinely generous
			but, fairly for their business, the insights cannot be used in paid products those not-for-profits might create

		the deeper issue is that the system is a black box
			you cannot audit how it works
			you cannot run it yourself
			and you certainly cannot offer it to clients or partners as part of a transparent workflow
		it is a remarkable research tool; but it isn't a foundation to build an organisation's independent capability on.
</pre>
  </main>
  <footer class="site-footer">
    <a class="back-link" href="/outline/hexagons_world/">Back to directory</a>
  </footer>
</body>
</html>
