Many proteins are transported into the endoplasmic reticulum by the universally conserved Sec61 channel. Post-translational transport requires two additional proteins, Sec62 and Sec63, but their functions are poorly defined. In the present study, we determined cryo-electron microscopy (cryo-EM) structures of several variants of Sec61-Sec62-Sec63 complexes from Saccharomyces cerevisiae and Thermomyces lanuginosus and show that Sec62 and Sec63 induce opening of the Sec61 channel. Without Sec62, the translocation pore of Sec61 remains closed by the plug domain, rendering the channel inactive. We further show that the lateral gate of Sec61 must first be partially opened by interactions between Sec61 and Sec63 in cytosolic and luminal domains, a simultaneous disruption of which completely closes the channel. The structures and molecular dynamics simulations suggest that Sec62 may also prevent lipids from invading the channel through the open lateral gate. Our study shows how Sec63 and Sec62 work together in a hierarchical manner to activate Sec61 for post-translational protein translocation.