Any foreigner who remembers their first time in Japan will remember it well. It’s not so much that we have entered somewhere we are not welcome, but rather a sense of finding ourselves in a kind of futuristic fairyland. It seems overwhelmingly strange, yet somehow makes us feel that we’re the strange ones. We suddenly realize that we look, speak, and smell differently from the majority, and for many, it’s the first time in our lives that we have been made hyper-aware of our own foreignness.

It’s natural to enjoy the attention at first—the Japanese are extremely welcoming to visitors in many cases, and Tokyo has no shortage of excitable enthusiasts who want to be best friends with the foreigner. The big problem with this is that you start to question whether they would look twice at you otherwise. Insecurity takes over, and what at first seemed like innocent friendliness starts to grate. We sulk when shop assistants respond in English when we speak to them in Japanese. We glare back at the obāsan who gawks at us from under her hat as she cycles past in the sun. We raise our eyebrows at our Japanese colleagues who utter an over-the-top “jōzu!” (“You’re so skilled!”) whenever we manage to use chopsticks at dinner without poking our own eyes out … which we want to do after hearing “jōzu!” for the 16th time. For better or worse, it’s inescapable: we are outsiders, and our identities here are constructed as such.

But being different doesn’t just give you license to whinge about how misunderstood you are; there are bonafide perks. I recently attended a music festival in Tokyo, and was initially thrilled to learn that foreign passport holders gained access for free. But as I stood in the relatively spacious “foreigners only” section beneath the front of the stage, mere meters away from Kyary Pamyu Pamyu, as hundreds of paying Japanese fans clamored against the barriers behind us, I started to feel like some kind of undeserving VIP imposter. Cue a lot of guilty gaijin dancing awkwardly to songs they didn’t understand the lyrics to.

Free gig tickets and a fragile conscience aren’t the only things you can cash in on for your gaijin sins. As any foreigner worth his salt knows, we inevitably get away with a lot that the Japanese don’t. Most expats have heard about, and likely orchestrated, the notorious “gaijin smash”—the act of a foreigner unknowingly or willfully breaking Japanese rules and conventions (of which there are many) with no real consequence. The fact that the rule-breaker is a gaijin usually trumps the rule-breaking act itself, because nobody can be bothered to argue with the hapless foreigner.

What’s interesting is how a lot of us react to being treated differently, and how much we complain about it. “Nothing really happens to us, but the passive-aggressiveness is definitely there,” goes the tired spiel. Funny that the white Westerners complaining about being marginalized are the same people who are idolized by the country’s media. That doesn’t mean Japan’s treatment of foreigners is right, but it should be seen in context with the wider world. No one in Japan is being assaulted or killed by police for the color of their skin, for example.

Of course, that’s not to ignore the difficulties that really do exist. It’s not unheard of—even in 2015—for foreigners to be refused gym memberships, employment, and apartments because they are not Japanese. These are old problems with no real new solutions. It is hoped the Olympics will bring about change, but 2020 is still a long way off and there’s a sense that these problems will sort themselves out.

Until then, and despite all the moaning, the gaijin’s world is still a privileged one of free drinks, special treatment, and heightened romantic attention (or so I’m told—that one remains a myth to me). When positive discrimination lives next door to problematic prejudice, where does the road lead? And if expats could truly be treated the same as Japanese people, would they really want to be? The bright-eyed foreigner might enjoy acting like a dyed-in-the-wool Tokyoite when it suits, but take away his magic gaijin glow and watch the disappointment in his face when nobody mentions those excellent chopstick skills.