Yasuhiro Yamashita was chosen to take over for Tsunekazu Takeda, who announced earlier this year he would step down at the end of his term. He also resigned from the International Olympic Committee, where he was the head of the powerful marketing commission.
Takeda, who led the JOC for almost two decades, has denied the corruption allegations that are being pursued by French investigators.
Yamashita won gold in judo at the 1984 Los Angeles Olympics and is one of Japan's most famous athletes. He is likely to be elevated to an IOC membership with the Tokyo Olympics opening in 13 months. He is also president of the All Japan Judo Federation.
The announcement came at the JOC's new headquarters, a $150 million facility built across from Tokyo's under-construction National Stadium.
Yamashita takes over as French authorities are set to try Lamine Diack and one of his sons — Papa Massata Diack — on corruption connected with the 2016 Rio de Janeiro Olympics and the 2020 Tokyo Games. The elder Diack was a powerful IOC member and is alleged to have received payments to sway IOC votes.
The scandal is a black eye for next year's Olympics, although they still seem to have widespread public support. Ticket demand in Japan is unprecedented, and Tokyo organizers have raised more than $3 billion in local sponsorship revenue — about three times as much as any previous games.
Much of that revenue has been driven by the giant Japanese advertising and marketing agency Dentsu Inc. Dentsu is the exclusive marketing agency for the Tokyo Olympics and has long-standing commercial links to both Diacks. French investigators have not suggested that Dentsu is a target of their probe.
Takeda was highly visible just a few months ago when he stood alongside IOC President Thomas Bach, exchanging small talk in Tokyo as Bach called the Japanese Olympics the "best prepared" in history.
Takeda has acknowledged he signed off on about $2 million in payments to a Singapore consulting company, which is believed to have been a conduit to the Diacks. Takeka, who has been largely portrayed as the fall guy, said he was not part of the decision-making process and had no reason to question signing a "regular commercial contract."
Takeda is a distant member of Japan's royal family, the great-grandson of the Meiji emperor who ruled late in the 19th century and into the 20th. He competed in equestrian events in the 1972 and 1976 Olympics.