According to my Mahayana training, saying that all things are empty (as a blanket statement per se) is wrong (nihilist). They are rather empty of inherent existence. It has to be qualified, as things do exist conventionally, just not ultimately. So the concept of inherent existence is crucial to this.
The problem is, nobody imposes such criteria on sensory data. Run of the mill, uninstructed people all acknowledge the fleeting nature of life.