What happens if, instead of doing a linear regression with sums of monomial terms, you do the complete opposite? Instead of regressing the phenomenon against , you regressed the phenomenon against an explanation like ?
I first thought of this question several years ago whilst living with my sister. She’s a complex person. If I asked her how her day went, and wanted to predict her answer with an equation, I definitely couldn’t use linearly separable terms. That would mean that, if one aspect of her day went well and the other aspect went poorly, the two would even out. Not the case for her. One or two things could totally swing her day all-the-way-to-good or all-the-way-to-bad.
If you don’t know my sister, you can pick up the point from this well-known stereotype about the difference between men and women:
"Men are simple, women are complex.” Think about a stereotypical teenage girl describing what made her upset. "It’s not any one thing, it’s everything.”
I.e., nonseparable interaction terms.
I wonder if there’s a mapping that sensibly inverts strongly-interdependent polynomials with monomials — interchanging interdependent equations with separable ones. If so, that could invert our notions of a parsimonious model.