3:86 -- Four species

This familiar midrash analogizes each of the four species in the lulav to a segment of the Jewish people -- the etrog, having taste and aroma, is like those Jews with both Torah learning and actual performance of mitzvot; the willow, having neither taste nor aroma, is like those Jews with neither Torah nor mitzvot, etc.

I'm puzzled by the assignment of the elements in the midrash. It says that the palm has taste but not aroma (the date doesn't have much aroma), and is like those Jews with Torah but no deeds, and the myrtle has aroma but no taste and is like those Jews with deeds but no Torah. I would have thought that aroma would be parallel to Torah (the spirit that pervades the physical but isn't physical), and taste would be parallel to deeds (the more apparently physical sense paralleling the physical deeds). Yet both in Leviticus Rabba and in Yalkut Shim'oni (the sources cited by Sefer HaAggada), it's the other way -- taste paralleling Torah, and aroma paralleling deeds. Why?