This is the perfect area for the history buff. Check out underground caverns with guided tours from Plain Road in the Golden Bay.
It made things easier if you speak their languages, hence a personal guided Taiwan easy and complimentary tour is generally recommended.