Eh . . .given what we've seen in the series, I don't think it matters since they are all the same and Naruto will retain all the memories when he disperses (assuming that hasn't been retconned too). Still, if that's the problem, the solution is for the "real" Naruto to stay home. Problem solved.
It's not whether he remembers. It's the principle of the matter. I think it's one thing that if he promises Bolt they'll got to the park only for Gaara to suddenly decide an invasion is nice because of some tainted ramen Naruto sent him. Life happens and you have to work. It's another matter to do it over and over. Sorry, son I'm busy, here's a shadow clone will only work so many times. If it happens a lot then the kid will feel like he's just being foisted off on a cheap copy. The act of being there and making the time matter otherwise he'll feel like second fiddle to work.