After receiving Mr Abe at the airport and taking him on an open jeep roadshow to Mahatma Gandhi's Sabarmati ashram, PM Modi escorted him to the iconic mosque built in the 16th century.
According to clerics at the mosque, PM Modi explained the history and importance of the mosque to Mr Abe. The two Prime Ministers were also photographed against the picturesque monument against the fading sunlight.
The famed lattice work of the mosque - the Tree of Life - has come to be the unofficial symbol of the city and has also inspired the logos of the National Institute of Design and the Indian Institute of Management-Ahmedabad (IIM).
The mosque was built by an Abyssinian, Sidi Saiyyed, who was from Yemen. It is said that about 45 craftsmen worked with him. Under the British, the monument mostly served the purpose of a government office.
In recent years, the 10-tone latticework windows have earned renown among historians and archeologists across the world. Officials see it as a symbol of the fusion of Hindu and Muslim cultures.
Mr Modi, who was Chief Minister of Gujarat for 12 years before taking power at the centre, visited the mosque today for the first time, at a time India is facing international criticism over alleged intolerance and amid accusations by the ruling BJP's critics of divisive politics.
PM Modi has in the past visited the Sarkhej Roza mosque and tomb complex on the outskirts of Ahmedabad.
On a visit to Abu Dhabi two years ago, he visited the Grand Mosque, the largest in the UAE.