I wouldn't throw a fit, either, and I probably wouldn't go to the trouble of returning the gift. I also wouldn't be too friendly with the person in the future.
Like I said, it's fine to assume but offensive to act on the assumption in a way like this before it's a certainty.