To investigate factors related to cognitive impairment among elderly residents of a town in Shimane Prefecture, we studied the cognitive status of 277 residents, aged 60 and older using Hasegawa’s Dementia Scale-Revised (HDS-R), and we assessed the residents’ activities of daily living, instrumental ADL, current occupation, self-rated health, medical history, and health-related behaviors using a questionnaire completed by the residents. We examined the cross-sectional association of cognitive impairment with the subjects’ answers to the questionnaire by logistic regression analysis. Among the 277 subjects, 20 were judged as cognitively impaired according to the HDS-R. In the self-rated health and disease categories, “falling risk,” “absence of vigor”(apathy), and “medical history of stroke” were significantly associated with cognitive impairment. We found that “absence of vigor” was significantly associated with cognitive impairment. It is important to screen for cognitive impairment in order to detect and prevent the development of dementia among the elderly.