This is a good point. Arthur, either willingly or subconsciously, always has a simplified (and somewhat idealized) version of Merlin in his head, which lets him treat Merlin without respect up until he actually needs Merlin's wisdom and guidance, but then he immediately reverts back to treating Merlin as he always has.
While I was watching the show for the first time, I thought Arthur treated Merlin like an emotional support teddy bear. The teddy bear is soft and comforting and keeps the nightmares away, and Arthur takes Merlin with him everywhere, even where he isn't needed, as a child with a favorite teddy bear might. But that teddy bear, in Arthur's eyes, is only an object, only there to comfort its owner, so why would he care about the teddy bear's wants or needs? It doesn't have any anyways!
This is why Arthur drags Merlin everywhere with him, regardless of what Merlin himself wants. And, like a favorite teddy bear, Merlin gets dirty, torn up, and damaged from the job of being there for Arthur, but Arthur never notices, too stuck focusing on what Merlin should do for him to see Merlin's current state.
Merlin is kind of like a dog to Arthur. I mean this in all of the good and all of the bad ways.