Learning four tones of Mandarin Chinese

From what I have experienced so far, it is neither the grammar nor even the pronunciation that makes speaking Chinese so difficult for native English speakers; it is the tonal nature of the language. The syllable ma, for example, means mother [mā (first tone)], hemp [má (second tone)], horse [mǎ (third tone)] or scold [mà (fourth tone)], and with neutral tone it is similar to a question mark, and the neutral tone sounds pretty much exactly like the first tone but just a shorter length, as far as I can tell. Here I am on my first day practicing Mandarin Chinese tones.