Legend has it that Bhairon Nath, a well-known Hindu Tantric, fell in love with a young Vaishno Devi after seeing her at an agricultural fair. However, Vaishno Devi was not interested in him and ran into the Trikuta hills to escape his advances. Later, she transformed into her original form of Durga and beheaded him with her sword in a cave. As per the author Manohar Sajnani, according to Hindu mythology, the initial dwelling place of Vaishno Devi was Ardh Kunwari, which is located about 6 km away from Katra town and the cave.

The Mahabharata, an ancient Indian epic, mentions the worship of goddess Vaishno Devi. Before the Kurukshetra War, one of the main battles in the epic, Arjuna sought the blessings of Durga on the advice of Lord Krishna. Impressed by his devotion, Mother Goddess appeared in front of him in the form of Vaishno Devi. Arjuna praised her with a stotra, a hymn of praise, which included the line 'Jambookatak Chityaishu Nityam Sannihitalaye', translating to 'you who always dwell in the temple on the slope of the mountain in Jambhu'. Scholars believe this refers to present-day Jammu. The Mata Vaishno Devi shrine is said to be an ancient site, even pre-dating the Mahabharata. Lord Krishna is believed to have advised Arjuna to seek the blessings of Vaishno Devi in the hills of 'Jambhu' before going to battle. 'Jambhu' is identified with present-day Jammu. Arjuna called Vaishno Devi the highest Yogin, free from decrepitude and decay, the Mother of the Vedas and the Science of Vedanta, and the giver and personification of victory itself. It is also believed that the Pandavas, the main characters in the Mahabharata, were the first to build the temples at Kol Kandoli and Bhawan in reverence and gratitude for the Mother Goddess. On a mountain adjacent to the Trikuta Mountain and overlooking the Holy Cave are five stone structures, believed to be the rock symbols of the five Pandavas.