InformerEventSource: trigger reconciles based on old and new resource state

[dvob]

Given you have multiple resources referencing each other, if these references do change, you have to trigger reconciles based on the old and new resource state to properly update the primary resources.

For example if you have two resources MyThing and MyThingConfig and multiple MyThingConfigs allows to configure a MyThing:

apiVersion: example.com/v1
kind: MyThing
metadata:
  name: my-thing
spec: {}
---
apiVersion: example.com/v1
kind: MyThingConfig
metadata:
  name: config-a
spec:
  myThingRef: my-thing
---
apiVersion: example.com/v1
kind: MyThingConfig
metadata:
  name: config-b
spec:
  myThingRef: my-thing

Now, if myThingRef changes, you want to trigger a reconciliation for the old and the new MyThing referenced by myThingRef so that MyThing properly reconciles with the new resulting configs.

See example code here: https://github.com/dvob/java-operator-sdk-repo-example/tree/4b9e01c3099ae580036aa81cbe1558c0d422f0b0 (including failing test).

As far as I understand controller-runtime (Go) solves this by triggering and event for the old and new resource: https://github.com/kubernetes-sigs/controller-runtime/blob/346f1930fde577d7d5e49c88e5ee625e4ebb7daa/pkg/handler/enqueue_mapped.go#L108

In the controller-manager built-in controllers there is a similar case with Services and Pods. If a Pod gets updated, it has to trigger reconciliation for the old and new Pod so that Services, which no longer select a Pod (old labels) can remove it and Services newly select a Pod (new labels) can add it.
https://github.com/kubernetes/kubernetes/blob/e64ed3d8ba86d0d6dc27c80710fc1570a9478f67/pkg/controller/endpointslice/endpointslice_controller.go#L547

@csviri already provided PR #3388 which supports this.
Also see related discussion in Discord https://discord.com/channels/723455000604573736/1508830486913482913

[csviri]

So there is a huge caveat for this (for the record), that controller might be down while the update is happening, so users should double check the state on each reconciliation (thus also on startup), to cover that.

[dvob]

So there is a huge caveat for this (for the record), that controller might be down while the update is happening, so users should double check the state on each reconciliation (thus also on startup), to cover that.

I do not fully understand why you consider this a huge caveat?

In my example, on restart every resource triggers an event either way, so all MyThings get reconciled and it doesn't matter if we miss updates from MyThingConfigs.

Another example I referenced are the K8s built-in Services. When the controller is running, on Pod updates we trigger a reconcile of all Services which select the old or new Pods. If the controller is down, on restart all services get reconciled either way.

[csviri]

I do not fully understand why you consider this a huge caveat?

Maybe it was too strong wording, mainly the intention was to articulate that for the record for newcomers, that there is this situation that what happens if controller is down; also that events might be lost in informers (although that might not impact this case)