The much-expected announcement by Mr Yeddyurappa came after a delegation of Karnataka BJP leaders met and formally invited him back to the party, setting the stage for the two parties to come together.
"We have decided to merge KJP with BJP," said Mr Yeddyurappa, flanked by senior BJP leader Ananth Kumar, who was perceived to be his bitter rival in Karnataka.
News agency PTI, quoting sources in the BJP, said the 70-year-old Karnataka strongman's return to the party is unconditional and he will not contest the Lok Sabha elections, due by May this year.
"All formalities will be completed within three or four days. From this minute, we both are going to work for winning more than 20 (of the 28) Lok Sabha seats in the state," Mr Yeddyurappa told PTI.
Ananth Kumar said with Mr Yeddyurappa's return, the BJP's strength in Karnataka would increase "ten-fold" and added that "good news" had been conveyed to party president Rajnath Singh who gave his "good wishes."
Mr Yeddyurappa's return was expected ever since Narendra Modi was named the BJP's presumptive PM in September last year. In November, two days before Mr Modi's rally in Bangalore, posters featuring Mr Yeddyurappa and Mr Modi together surfaced in the city.
The 70-year-old politician, a known Modi loyalist, had declared that the Gujarat chief minister should be the prime ministerial candidate over party veteran LK Advani, who led a section in the BJP that insisted Mr Yeddyurappa resign as Chief Minister over charges of corruption. The case against him is still pending.
A powerful leader from the influential Lingayat community, Mr Yeddyurappa won the BJP its first government in south India six years ago.