Mr Chouhan offered prayers at the Priyakant Ju temple Saturday evening.
He also inaugurated a gallery portraying various activities of Lord Krishna as a child.
Founder of the temple Devkinandan Thakur said, "After listening to a few devotional songs, the chief minister took the mike from me, and to my astonishment, started singing 'bhajans'."
"Singh was overwhelmed by the ambience of the temple," Mr Thakur said.
The chief minister did not interact with reporters during his stay here. "I am on pilgrimage," he told reporters.