The simple answer to this is "no". Although I do speak Japanese, I have colleagues who have happily lived in Tokyo for many years with no Japanese at all. The key here is the word Tokyo though. Outside Tokyo, the English ability disappears very rapidly and at least basic Japanese becomes much more important. Even in Osaka, a city of more than eight million people, English is very thin on the ground.
If you are thinking of coming to Japan to work, your company may provide Japanese lessons. These may be of help but the real learning comes with practicing with Japanese friends, and that doesn't come overnight. Setting realistic targets such as how to direct a taxi or asking simple directions will take a lot of the strain out of living in Japan and will help direct actually learning the language.
If you're on a 2~3 year assignment, you're not going to working in Japanese unless you're a good linguist to start with. And then the answer may be in keigo (the honorific vocabulary) and you may just be lost all over again. Investing time into understanding Japanese business practices and cultural approaches will provide rich rewards over an above learning the language itself.
However, learning the language is enjoyable in its own right and really can be very rewarding as the fog of life begins to clear and everything becomes just a little less confusing.