Over the last 25 years, a large number of studies have looked at the demographic differences between the previously divided eastern and western parts of Germany. The two regions have converged with respect to mortality and overall fertility levels. But in terms of family formation behaviour, differences remain. Non‐marital births are the norm in eastern Germany, whereas they are still the exception in western Germany. Various explanations for these differences have been offered, with the most persuasive linking policy and socio‐economic conditions in eastern and western Germany after 1945 with the persistence of regional patterns. Here, we show that the non‐marital fertility divide pre‐dates the 1945 division of Germany. Indeed, already in the early 20th century, the areas of eastern Germany that made up the German Democratic Republic had, on average, roughly twice the non‐marital fertility levels of western Germany. In the first part of our paper, we document the history of this long‐standing pattern and provide a set of explanations for its emergence. In the second part, we apply multi‐level models to birth register data to explore whether East–West differences in non‐marital fertility would remain even under scenarios such as convergence in secularisation levels and economic conditions. The persistence of the past suggests that explanations for family formation differences between eastern and western Germany based solely on the most recent and current conditions are incomplete, and that convergence, if it occurs at all, will take longer than anticipated, perhaps lasting many decades or more. Copyright © 2014 John Wiley & Sons, Ltd.