Household food insecurity is a pervasive problem in North America with serious health consequences. While affordable housing has been cited as a potential policy approach to improve food insecurity, the relationship between conventional notions of housing affordability and household food security is not well understood. Furthermore, the influence of housing subsidies, a key policy intervention aimed at improving housing affordability in Western countries, on food insecurity is unclear. We undertook a cross-sectional survey of 473 families in market rental (n = 222) and subsidized (n = 251) housing in high-poverty urban neighborhoods to examine the influence of housing circumstances on household food security. Food insecurity, evident among two thirds of families, was inversely associated with income and after-shelter income. Food insecurity prevalence did not differ between families in market and subsidized housing, but families in subsidized housing had lower odds of food insecurity than those on a waiting list for such housing. Market families with housing costs that consumed more than 30% of their income had increased odds of food insecurity. Rent arrears were also positively associated with food insecurity. Compromises in housing quality were evident, perhaps reflecting the impact of financial constraints on multiple basic needs as well as conscious efforts to contain housing costs to free up resources for food and other needs. Our findings raise questions about current housing affordability norms and highlight the need for a review of housing interventions to ensure that they enable families to maintain adequate housing and obtain their other basic needs.