Discovery of coding variants in genes that confer risk of neurodevelopmental disorders is an important step towards understanding the pathophysiology of these disorders. Whole-genome sequencing of 31,463 Icelanders uncovers a frameshift variant (E712KfsTer10) in microtubule-associated protein 1B (MAP1B) that associates with ID/low IQ in a large pedigree (genome-wide corrected P = 0.022). Additional stop-gain variants in MAP1B (E1032Ter and R1664Ter) validate the association with ID and IQ. Carriers have 24% less white matter (WM) volume (β = -2.1SD, P = 5.1 × 10-8), 47% less corpus callosum (CC) volume (β = -2.4SD, P = 5.5 × 10-10) and lower brain-wide fractional anisotropy (P = 6.7 × 10-4). In summary, we show that loss of MAP1B function affects general cognitive ability through a profound, brain-wide WM deficit with likely disordered or compromised axons.