You mean you don't like me?

If you have ever had a discord with a person because they misunderstood what you were saying, maybe this article from the Linguistics Society of America will help put things in perspective.

Although the article talks about different cultures understanding everyday events differently, the thinking can be extrapolated to the point where every person's 'personal culture' is slightly different from every other person. Your personal culture depends on events and experiences that are unique to you. This would also explain why that childhood buddy understands you so well while your latest boy/girl friend doesn't. Also, to a slight extent (or maybe to a large one), it explains why men and women behave and act differently in different situations - the collective personal cultures of the genders are different as they mature.

Although the article mainly focuses on language and whether the different language speakers see the world differently, from the above argument one can see that the difference is not only from the language you speak, but also from the unique cultural and personal experiences you have had.

If you have any views on the post, do give feedback!