The Alchon Hun king, Mihirakula is said to have been the founder of the temple, dedicated to Shiva, a shrine named Mihiresvara in Halada, and a large city called Mihirapura. In 1899 James Fergusson placed the temple construction to the 17th to 18th century. Fergusson disputes claims that structures on the basis of which he makes this claim are from repairs. Aurel Stein while agreeing that the superstructures are from a more recent date, places the base and the stairs as much older. Historical figures associated with the monument include Jaloka, one of the sons of Ashoka (Gonandiya), according to the Rajatarangini. Kashmiri Hindus strongly believe the temple was visited by Adi Shankara and has ever since been associated with him; this is how the temple and hill got the name Shankaracharya. It is here that the literary work Saundarya Lahari was composed. It was composed by Adi Shankara after accepting the major faith in the region at the time, that of Shakti, and that the union of Shiva and Shakti, as in Shaktism, transpired into the symbolism of the Sri Yantra.

Though the exact date of construction of this temple is not known, people believe that Jaluka, the son of the Mauryan Emperor Ashoka, built it in 200 BC. Sultan Sikandar, the ruler of Kashmir, destroyed the Shiva Lingam of the temple and several other idols of Gods. Later, different rulers at different times restored the temple because of the natural disasters and several attacks. According to popular belief, several kings have repaired this temple over a period. The temple was rebuilt and renovated by King Godapatta in the first century AD, and he dedicated this temple to Jyeteshwara. King Gopaditya (426-365 BC) and King Lalitaditya (697-734 AD) carried out later renovations. Gulab Singh of the Dogra dynasty renovated the Durga Naag shrine on its premises. The King of Mysore installed electric fittings in the temple in the modern period. In 1961, the Shankaracharya of Dwarkapeeth installed the statue of Adi Shankaracharya. The current structure of the temple traces its origin to the 9th century AD during the reign of Sikhs. It was when the Shivalinga was placed in the temple. The Sikh rulers revived the fortunes of this temple with regular prayers and festival celebrations. The temple is also called Bagh-i-Sulaiman by the Jews and the Persian community. Evidence indicates that this temple belongs to ancient times as scholars have found various Persian scriptures here.