This study examines the determinants of forest and tree-product uses in rural households across three sites of different proximity to roads and forests in the Chittagong Hill Tracts region in Bangladesh. A structured questionnaire survey was conducted with 300 households of different ethnic groups, located in three different locations (remote, intermediate, on-road), to collect information on their forest and tree use during 2015–2016. We gathered information on household socioeconomic characteristics (family size, education level of head of household, size of farmland), location (three sites), and ethnic affiliation. By conducting a series of logistic regression modeling, we analyzed the key determinants that would explain the variations in forest use in the households. We recorded twelve different forest and tree products used in the households, primarily for subsistence purposes and cash income. Fuelwood, vegetables, and fish were recorded as the most important forest-sourced products used by people, regardless of socioeconomic condition, location context, and ethnic affiliation. Household land/farm size, location, and ethnic background explained significant variations in the use of forest and tree products (mainly timber, fodder for livestock). The greater the size of the landholding, the more likely timber was used for both subsistence and cash income, but the less the reliance on other products (fuelwood, thatch grass, vegetables). Our findings suggest that the location and ethnic characteristics of the rural households are important for understanding the diverse needs for forest and tree use, and should be factored into the site-specific management and sustainable use of forest and tree resources in Bangladesh and other tropical developing countries.