Couple of weeks is perfect.
Get a rail pass, don't stay in Tokyo the entire time. One week in Tokyo is plenty to see & do a ton. At the very least do day trips to places 1-2 hours away. Take a short mountain hike somewhere while you are there. Like, doesn't have to be climbing a mountain solo, there's lots of places that are like right outside town, you get off a train and follow everyone along a scenic fresh air path somewhere.
I'd absolutely recommend staying at a hot springs inn (Ryokan) for at least 1 night, if not 2. The smaller the inn, the better because it can be like your own private place where you never run into anyone else. It took me like 4-5 trips of Japan before I did this and now and for the rest of my life I spent 2 nights at a Ryokan every time I go to Japan. It's the best thing ever. The experience, the FOOD (most will provide breakfast & dinner), the hot springs...it's the best. The people running them are usually wonderful and a lot speak English! Especially if they have an English website.
If you're interested, I can recommend my favorite one and give you tons of info on it, it's called Jinpyokaku. You can get there from Tokyo in about 3.5 hours and in March it'll be cold, but should be past the snow. If you go there you can let them know when you're arrive at Yudanaka (you take a train from Tokyo -> Nagano and then change lines to go from Nagano -> Yudanaka) and they'll pick you up in a van from the station and drive you up the hill about 20 mins to the ryokan. During the day you can walk through a cool forest to the Snow Monkey park and check out the monkeys. Also sitting in the outdoor hot springs bath in the cold night all by yourself just chilling as leaves fall down and stuff is amazing. Only way it could be better is if you're sipping sake while doing it, but I haven't tried sneaking sake in...yet.
http://www.jinpyo.jp/eng/But there's like 1,000 ryokans in Japan, so you can find others and ones closer or somewhere along your way. But I would recommend spending 2 nights at a Ryokan somewhere on your trip. It might be your favorite memory/part.
Otherwise, you can day trip to Kamakura, Nikko, Enoshima island, Yokohama, Hakone (I didn't think Hakone was worth it). Just google around the places and find stuff you think would be fun to visit. If you're going to leave Tokyo, then Osaka/Kyoto is good. Osaka will be pretty close to the same Tokyo experience but with Chronovore. The Aquarium is amazing. Kyoto is pretty unique and you can spend a day or two visiting temples or the train museum or the international manga museum.
My recommended itinerary for Tokyo is:
-Akihabara game shopping & Club Sega for your arcade gaming
-Ueno park museums
-Yoyogi Park & Harajuku on Sunday around lunch in case cosplayers are on the bridge.
-Tokyo station for the old Tokyo station exit which is historic and the underground shops including the brand shops like the Kirby shops; also you can walk to the imperial grounds and walk around the outside or look for a tour thing for inside.
-Odaiba for the Gundam and Amusement park rides and museums
-Tokyo Tower/Skytree are probably cool, I suck and still haven't been up either
-Sunshine City in Ikebukuro has some anime/manga stuff like Mandarake and Namco city
-Shinjuku, Kabukicho is the red light district that the Yakuza games Kamurocho is based on, there's also GODZILLA there