Current in vivo neuroimaging techniques provide limited field of view or spatial resolution and often require exogenous contrast. These limitations prohibit detailed structural imaging across wide fields of view and hinder intraoperative tumor margin detection. Here we present a novel neuroimaging technique, speckle-modulating optical coherence tomography (SM-OCT), which allows us to image the brains of live mice and ex vivo human samples with unprecedented resolution and wide field of view using only endogenous contrast. The increased visibility provided by speckle elimination reveals white matter fascicles and cortical layer architecture in brains of live mice. To our knowledge, the data reported herein represents the highest resolution imaging of murine white matter structure achieved in vivo across a wide field of view of several millimeters. When applied to an orthotopic murine glioblastoma xenograft model, SM-OCT readily identifies brain tumor margins with resolution of approximately 10 μm. SM-OCT of ex vivo human temporal lobe tissue reveals fine structures including cortical layers and myelinated axons. Finally, when applied to an ex vivo sample of a low-grade glioma resection margin, SM-OCT is able to resolve the brain tumor margin. Based on these findings, SM-OCT represents a novel approach for intraoperative tumor margin detection and in vivo neuroimaging.