Demographic aging is a worldwide phenomenon, cognitive and behavioral impairment is becoming global burden of nerve damage. However, the effect of pharmacological treatment is not satisfying. Therefore, we analyzed the efficacy of music therapy in elderly dementia patients, and if so, whether music therapy can be used as first-line non-pharmacological treatment. A comprehensive literature search was performed on PubMed, EMbase and the Cochrane Library from inception to September 2016. A total of 34 studies (42 analyses, 1757 subjects) were included; all of them had an acceptable quality based on the PEDro and CASP scale scores. Studies based on any type of dementia patient were combined and analyzed by subgroup. The standardized mean difference was -0.42 (-0.74 to -0.11) for disruptive behavior and 0.20 (-0.09 to 0.49) for cognitive function as primary outcomes in random effect models using controls as the comparator; the secondary outcomes were depressive score, anxiety and quality of life. No evidence of publication bias was found based on Begg's and Egger's test. The meta-analysis confirmed that the baseline differences between the two groups were balanced. Subgroup analyses showed that disease sub-type, intervention method, comparator, subject location, trial design, trial period and outcome measure instrument made little difference in outcomes. The meta-regression may have identified the causes of heterogeneity as the intervention method, comparator and trial design. Music therapy was effective when patients received interactive therapy with a compared group. There was positive evidence to support the use of music therapy to treat disruptive behavior and anxiety; there were positive trends supporting the use of music therapy for the treatment of cognitive function, depression and quality of life. This study is registered with PROSPERO, number CRD42016036153.