Inzamam, whose tenure ends on July 31, said that critics and some former players focussed merely on criticising his nephew.
“Imam, I think would not have survived for so long in the Pakistan team and international cricket if he was not a performer. He had already played junior cricket for Pakistan and done well in domestic cricket even before I became chief selector but this fact was ignored by the critics which was hurtful for me,” he said.
“Being chief selector has been the most challenging role of my cricket career as I was heavily criticised for my decisions and I couldn’t respond much,” he added.
The 23-year-old Imam has played 10 Tests and 36 ODIs for Pakistan and his selection has often been criticised by ex-players.
The former captain and batting mainstay conceded that working as chief selector has been tough.
Inzamam, who appeared in 120 Tests, stepped down after a three-year tenure which ended with Pakistan not qualifying for the semi-finals of the World Cup in England.
But during his tenure, Pakistan won the Champions Trophy in 2017 in England and also became world number one in the ICC T20 rankings.
Inzamam said cricket is his life and he would always be involved with the sport in some capacity even after stepping down as chief selector.
“I can’t leave cricket it is my passion,” he said.
Inzamam, who came under fire for going to England before the World Cup and getting directly involved in team matters, said he had gone there with the permission of the Pakistan Cricket Board.
“I did my best to help them out. Our team didn’t do badly but luck didn’t favor us many times. As chief selector I did my best to pick new talent and give them proper opportunities as they are the future of Pakistan cricket,” he said.
Inzamam is tipped to get a fresh assignment when the cricket board sits down next week to review the performance of the team in the last three years.
Head coach, Mickey Arthur is scheduled to give a presentation to the cricket committee next Monday.
Pakistan plays nine Tests, nine ODIs and some 16 T20 internationals in the run-up to the ICC World T20 Cup in Australia in October-November next year.