Stable Constraints Make Learning Possible—Incompleteness Keeps the World Open

Intelligence Navigates by Invariants

Contents

Navigation is usually described as movement through a space. Yet movement alone is not navigation. A system must be able to distinguish one state from another, preserve some relation between observations and recognize whether a step has brought it closer to or farther from a goal.

That requires something to remain stable.

A ship navigates because stars, gravity, magnetic fields and the laws of motion behave predictably enough to provide reference. A person navigates a city because streets, landmarks and spatial relations persist between observations. A language model navigates a learned distribution because statistical regularities connect tokens, concepts and contexts.

This suggests a broader proposition:

Navigation is possible only relative to invariants.

Not invariants in the absolute sense of things that can never change, but relationships stable under the transformations relevant to the task.

An invariant is a property or relationship that remains unchanged under a specified transformation. The qualification matters. Distance is preserved under a rotation in Euclidean geometry, but not under every possible transformation. Energy is conserved under conditions described by a physical theory. A software invariant holds at a defined point in a program, not everywhere in the universe.

Felix Klein’s Erlangen programme framed geometries through the properties preserved by groups of transformations. Emmy Noether later established a profound relationship between continuous symmetries and conservation laws in physics. In both cases, invariants make change intelligible: they tell us what counts as the same structure while something else varies.

Real navigation uses several kinds of stability:

Invariant typeNavigational function
PhysicalSupplies persistent relations such as inertia or conservation
LogicalPreserves validity across permitted transformations
StatisticalSupports generalization across observations and environments
CausalIdentifies relationships expected to survive intervention
NormativeKeeps decisions inside chosen values, rights or policies
OperationalMaintains requirements, identity and state across a process

These are not interchangeable. A statistical regularity is not a physical law, and a policy is not discovered in nature. They share a functional role: each supplies a reference against which change can be interpreted.

Invariants Create the Information Landscape

Navigation as knowledge requires more than access to facts. It requires a landscape with distinguishable directions.

Invariants help create that landscape by establishing equivalence and difference. Two observations can be treated as instances of the same pattern because some relation survives their differences. Other changes become consequential because they violate what was expected to remain stable.

This also connects navigation to the entropy budget of intelligence . Entropy describes uncertainty across possible states. Invariants do not simply eliminate that uncertainty. They organize it. They partition possibilities into classes, create constraints and make gradients of relevance visible.

A completely random sequence may carry high Shannon information because its next symbol is difficult to predict. But if no reusable relation persists across observations, it offers little basis for learning a route. Information can exist without a pattern that supports robust navigation.

The stronger claim is therefore not “without invariants there is no information.” It is:

Without sufficiently stable relations, information cannot become reusable orientation.

Intelligence Is Not the Possession of Invariants

If environments never changed, a fixed rulebook might be enough. Intelligence becomes necessary because useful invariants are rarely given in advance and rarely hold without qualification.

A more useful working definition is:

Intelligence is the capacity to discover, test and revise invariants for navigation in an incomplete world.

This definition includes several operations:

  1. Notice patterns that survive variation.
  2. Distinguish stable relations from accidental correlations.
  3. Identify the transformations under which a relation remains valid.
  4. Use that relation to predict or choose a path.
  5. Revise it when the environment produces a counterexample.

Machine-learning research on invariant risk minimization makes one version of this problem explicit. It seeks predictors that remain useful across multiple training environments rather than exploiting correlations specific to one distribution. The ambition is directly navigational: learn a representation that continues to orient the model when the surroundings change.

The practical methods do not solve the general problem. Subsequent work has shown that proposed formulations can fail to recover the desired invariances even in simple settings. Discovering the right invariant is itself an uncertain search.

Selection Always Embeds an Invariant

The two engines of discovery —variation and selection—depend on invariants in different ways.

Variation changes candidates. Selection asks whether something important survived the change.

A code optimizer may vary an implementation while preserving correctness. A scientific explanation may change while remaining consistent with observed evidence. A design may be simplified while preserving its required function. An aligned system may explore different actions while remaining inside a safety constraint.

Every evaluator therefore encodes a claim about invariance:

  • this test must continue to pass;
  • this evidence must still be explained;
  • this right must not be violated;
  • this identity must remain traceable;
  • this objective must survive optimization.

Selection without an invariant has no basis for preferring one transformation over another. But selecting the wrong invariant is equally dangerous. A system can preserve benchmark performance while losing fairness, preserve engagement while degrading well-being, or preserve an answer format while abandoning truth.

Learned Invariants Can Be Wrong

LLMs learn statistical regularities across enormous corpora. Those regularities provide powerful orientation: syntax survives changes in subject, concepts recur across phrasings and familiar relations support useful analogies.

But frequency is not necessity. A model can treat a historical bias as stable, a temporary convention as universal or a spurious correlation as causal. What looks invariant inside the training distribution may fail under a new culture, domain, time period or adversarial case.

This is why invariants must remain falsifiable. A robust knowledge system records their scope, provenance and exceptions. It distinguishes among:

  • discovered invariants, inferred from observation;
  • formal invariants, guaranteed inside a defined system;
  • engineered invariants, enforced by architecture or tests;
  • chosen invariants, preserved because people decide they matter.

Confusing these categories turns orientation into dogma. Treating a chosen value as a natural law hides responsibility. Treating a learned pattern as universal hides uncertainty.

What Gödel Does—and Does Not—Show

Gödel’s incompleteness theorems concern sufficiently expressive, consistent, effectively axiomatized formal systems capable of representing a certain amount of arithmetic. They show that such a system contains statements it can neither prove nor disprove using only its own rules; the second theorem limits its ability to prove its own consistency.

They do not prove that every map, AI system or account of reality is incomplete. Applying the theorems directly to minds, societies or general information spaces requires additional arguments. Many popular extensions go beyond what the formal results establish.

Gödel can still play a disciplined role here: not as proof, but as motivation and boundary case.

A formal system navigates derivations relative to axioms and inference rules. Gödel shows that, for an important class of such systems, a stable formal framework does not guarantee complete internal reach. The framework provides orientation precisely while leaving some questions unresolved within it.

That combination is philosophically suggestive: stable orientation does not require a final, complete map.

The open research question is broader than Gödel:

Under what conditions must navigation through an open information space rely on invariant systems that remain incomplete or revisable?

Stable Orientation in an Open World

Incompleteness and invariance are not opposites.

Invariants make navigation possible. Incompleteness makes navigation necessary. If a system possessed a final map containing every relevant state, transformation and consequence, it would not need to learn in the ordinary sense. It could retrieve the complete route.

Real intelligence works differently. It preserves enough structure to act while remaining capable of discovering that its structure was local, approximate or wrong.

This suggests a design principle for AI: do not ask a model to behave as if every learned regularity were permanent. Build systems that can expose which invariants they are relying on, identify their scope, test them across changing environments and escalate when they fail.

The goal is not a world without uncertainty or a system with an invariant for everything. It is a navigational intelligence that knows what it is holding fixed, why it matters and when the surrounding world has changed enough to let it go.

Sources: