Is it this fuzzy blob of ineffable experiences, aesthetics, qualia, self-reference, dualism, spirituality and other things? I don't need to reject all this to notice that its fuzziness makes it impossible to operationalize.
No... again a misunderstanding of the word, and a major one!
A soul - and I mean this in the most atheistic, nonreligious sense you can imagine - is what you operationalize every single day when you're faced with hardship or any situation where you'd prefer not to be. It's also the dynamic that makes you feel joy in doing something beyond simple pleasure or happiness.
You know SQL? Think of "soul" as a view of the tables describing your intellectual and physical properties, left joined with your actions, which filters those actions that would otherwise not be simply mechanical or deterministic. Does that help?
Is it that complex? Whatever you feel stays inside you. From the outside you make noises, gestures, facial expressions. We can obtain fMRI data an so on. Operationalization is how to connect observables with the presence of soul in a subject.
What can you say besides "I don't feel a soul inside an LLM"? You don't feel it? OK, good for you, whatever.
