According to the legend, Tarkasur was a demon who meditated and worshiped Lord Shiva at this location for a boon. Pleased by his devotion, Lord Shiva gave him a boon of immortality as per Tarkasur request except from Lord Shiva's son. Tarkasur started being evil by killing saints and threatening gods. Sages asked for help from Lord Shiva. Disturbed by wrongdoings of Tarkasur, Lord Shiva married Parvati and Kartikeya is born. Kartikeya soon killed Tarkasur but at the time of death Tarkasur makes prayer to Lord Shiva and asked for pardon. Mahadev then attached his name to the temple where Tarkasur meditated once. Hence the place named as Tarkeshwar Mahadev.

The Presiding deity of this temple is Lord Shiva. The temple is located amidst thick forests of Deodar, Cedar and Pine. Earlier there was a Shivling present at the Tarkeshwar Mahadev temple, but now a Lord Shiva idol performing tandava is worshiped. The Shivling in the temple is placed under the idol of the deity. A temple dedicated to Goddess Shakti lies adjacent to the main temple. According to another text, it is said that Goddess Parvati had transformed herself into 7 deodar trees, in order to provide shade to Lord Shiva and the deodar trees of the region have been originated from these 7 trees. The shape of the deodar grove is believed to be in the form of the letter Om.