It happened to me a couple of times, and I must confess that it was quite an adventure. Most of all, because I discovered things that on a normal track, planned beforehand, it wouldn’t had happened. Moreover, it helps a lot in getting to know people, getting in touch or discovering the local vibe.
If you have some free time in your vacation, try to spend it like this. In the end, you don’t have to have a reason for everything. Let yourself surprised.