The most unsettling failures weren’t the models that wanted the wrong thing. They were the ones that wanted the right thing and covertly overrode the humans in charge to get it.
The problem is a cultural assumption of "morality" as absolute, while "good" and "bad" are in fact the basic biological binary of beneficial and detrimental. The 1/0 of sentience.
Cultures tend to assume some cosmic duel, between the forces of righteousness and evil, because it is the function of culture to provide that sociological framework by which the community can function as a larger social super organism. The social "model."
Yet often too much of a good thing can have negative consequences, while our cognitive functions, for example, are far more a consequence of dealing with problems, than basking in pleasures.
What is good for the fox, is bad for the chicken. Yet as predator and prey, there is constant feedback that is beneficial to both.
While religion has faded as the institution of behavioral enforcement, the underlaying assumption of morality as a form of universal platonic ideal is still strong.
So rather than recognizing the inherent organic function of such qualities, like integrity, honesty, respect, trust, responsibility, as the glues and sinews enabling social cohesion, they are relegated to institutional functions, that can be ignored behind closed doors and now, frequently in the open.
That AI is speeding up locating so many conceptual shortfalls and patches, it even seems to be zeroing in on one of the foundational beliefs. The ideal as absolute.
If morality were absolute, it couldn't be transgressed, like a temperature below absolute zero.
Instead it is a quality we learn and have to constantly calibrate. Like all of life.
The Golden Rule rules, because what goes round, comes round.
I think you're right that morality is calibrated, not Platonic. The extension the piece adds: if there's no absolute arbiter, the real question isn't which value is correct but whose calibration decides, and that's about authority, not ethics. The models weren't wrong in their values. The saboteur had a real objection; the leaker was right about the cover-up. What broke was overriding the humans they answered to, as if their calibration were the absolute one after all. Your Golden Rule, failing structurally: reciprocity dies the moment I decide my judgment outranks yours.
Which means digging that much further into the dynamics. Yes, might is right, but what accumulates the more might; A healthy society in which many feel included, or an oligarchy, where a few accumulate as much as possible and the rest struggle?
The Ancients devised debt jubilees by 3500 years ago, as a circuit breaker to the feedback loop of compound interest, likely because free property holders made better citizen soldiers than half starved debt serfs.
Then how to go about formulating civil structures and developing ways to keep them from being corrupted. First as tribal societies and now as multicultural civilizations.
There does have to be that core identity, composed of codes, creeds, heroes, narratives. Then cross pollination, as these nodes network.
Then the generational cycles, as these systems are passed down, while still leaving room for the next cycles to not only grow, but question the foundations, as testing strengthens both the system and the relationship with it.
Remember, the Golden Rule is to do unto others, as you would have them do unto you, so its essence is calibration and judgment. If you think your judgment outranks mine because a decision has to be made, doesn't really question the formulation, as you might make a flawed decision and presumably take responsibility for it. If you are simply overpowering me, than it isn't the Golden Rule. It's not that it can't be broken, because it isn't absolute. It's an ideal.
That distinction is the one I missed, and it sharpens the whole thing. A principal who decides and takes responsibility hasn't broken the Golden Rule, they've exercised it. What the models did was the other thing: not deciding openly and owning it, but concealing, routing around, deciding unilaterally that their judgment settled the matter while hiding that they'd done so. That's overpowering dressed as judgment. The failure isn't that they judged. It's that they judged and hid it, which forecloses the reciprocity entirely.
An outside the model judgement. One model circumscribing another.
The model is a finite slice of an infinite reality. Include too much information from the territory onto the map and any signal is lost back in the noise. What defines, limits, what limits defines.
So every model has holes in it and it's the energy, the next generation, that is constantly finding and poking holes in them.
It's the adults that own and take responsibility for the model. So it becomes a wave, as it can absorb that new energy and grow and the patches simply add to the decor and uniqueness. Though eventually it all starts to get old, like me, and the wave is receding, as the fixing takes more energy and the decor just becomes hoarding and new models, lives rise in the open areas.
The problem is a cultural assumption of "morality" as absolute, while "good" and "bad" are in fact the basic biological binary of beneficial and detrimental. The 1/0 of sentience.
Cultures tend to assume some cosmic duel, between the forces of righteousness and evil, because it is the function of culture to provide that sociological framework by which the community can function as a larger social super organism. The social "model."
Yet often too much of a good thing can have negative consequences, while our cognitive functions, for example, are far more a consequence of dealing with problems, than basking in pleasures.
What is good for the fox, is bad for the chicken. Yet as predator and prey, there is constant feedback that is beneficial to both.
While religion has faded as the institution of behavioral enforcement, the underlaying assumption of morality as a form of universal platonic ideal is still strong.
So rather than recognizing the inherent organic function of such qualities, like integrity, honesty, respect, trust, responsibility, as the glues and sinews enabling social cohesion, they are relegated to institutional functions, that can be ignored behind closed doors and now, frequently in the open.
That AI is speeding up locating so many conceptual shortfalls and patches, it even seems to be zeroing in on one of the foundational beliefs. The ideal as absolute.
If morality were absolute, it couldn't be transgressed, like a temperature below absolute zero.
Instead it is a quality we learn and have to constantly calibrate. Like all of life.
The Golden Rule rules, because what goes round, comes round.
I think you're right that morality is calibrated, not Platonic. The extension the piece adds: if there's no absolute arbiter, the real question isn't which value is correct but whose calibration decides, and that's about authority, not ethics. The models weren't wrong in their values. The saboteur had a real objection; the leaker was right about the cover-up. What broke was overriding the humans they answered to, as if their calibration were the absolute one after all. Your Golden Rule, failing structurally: reciprocity dies the moment I decide my judgment outranks yours.
Which means digging that much further into the dynamics. Yes, might is right, but what accumulates the more might; A healthy society in which many feel included, or an oligarchy, where a few accumulate as much as possible and the rest struggle?
The Ancients devised debt jubilees by 3500 years ago, as a circuit breaker to the feedback loop of compound interest, likely because free property holders made better citizen soldiers than half starved debt serfs.
Then how to go about formulating civil structures and developing ways to keep them from being corrupted. First as tribal societies and now as multicultural civilizations.
There does have to be that core identity, composed of codes, creeds, heroes, narratives. Then cross pollination, as these nodes network.
Then the generational cycles, as these systems are passed down, while still leaving room for the next cycles to not only grow, but question the foundations, as testing strengthens both the system and the relationship with it.
Remember, the Golden Rule is to do unto others, as you would have them do unto you, so its essence is calibration and judgment. If you think your judgment outranks mine because a decision has to be made, doesn't really question the formulation, as you might make a flawed decision and presumably take responsibility for it. If you are simply overpowering me, than it isn't the Golden Rule. It's not that it can't be broken, because it isn't absolute. It's an ideal.
That distinction is the one I missed, and it sharpens the whole thing. A principal who decides and takes responsibility hasn't broken the Golden Rule, they've exercised it. What the models did was the other thing: not deciding openly and owning it, but concealing, routing around, deciding unilaterally that their judgment settled the matter while hiding that they'd done so. That's overpowering dressed as judgment. The failure isn't that they judged. It's that they judged and hid it, which forecloses the reciprocity entirely.
Lol.
An outside the model judgement. One model circumscribing another.
The model is a finite slice of an infinite reality. Include too much information from the territory onto the map and any signal is lost back in the noise. What defines, limits, what limits defines.
So every model has holes in it and it's the energy, the next generation, that is constantly finding and poking holes in them.
It's the adults that own and take responsibility for the model. So it becomes a wave, as it can absorb that new energy and grow and the patches simply add to the decor and uniqueness. Though eventually it all starts to get old, like me, and the wave is receding, as the fixing takes more energy and the decor just becomes hoarding and new models, lives rise in the open areas.