This small park is the site of a former Japanese temple, which was built at the end of the 19th century for the country's military, and which offered medical and spiritual care for wounded and dying soldiers. With the arrival of the KMT in 1949, it was converted into living quarters for Chinese soldiers. A fire in 1975 destroyed the temple, leaving only the shell of the mausoleum and the Rinbansyo, now a lovely teahouse, where the head priest lived.
There's not a lot to see here except a small hillock with a wooden bell tower. The building next to the fleet of steps where the main temple once stood is the Taipei City Archives (open 9am to 5pm weekdays); it has English language materials and a map of the square.