but with a more powerful "AI" ?
Firstly, no robotics kit has any AI built in. AI needs to be tailor-made for the situation, it's not like you
import AI and everything just... works. You need to program your own AI every time you want a smart bot. That being said, some AI-related things like image processing can be found prepackaged for use.
Regarding kits, I don't know of there's any such kit that's better than the NXT. For "serious robotics", such kits will always prove inadequate as they're restrictive in many ways. What I suggest is to start using an Arduino or Raspberry Pi. A Raspberry Pi fits your criteria more--it has a good amount of RAM, and runs Linux--so it's easier to work with (it's pretty much like programming a normal application, except you don't need to create a GUI for the app). It's extremely cheap, costs $25-$35. You can use whatever language you want here, as long as you have something to make the language run on Linux (gcc/python/java--generally these are all built in, though you can easily find APT packages for stuff like ruby). In addition, it's easy to find packages for tasks like image processing/etc. It also has 512MB of RAM, which is good enough unless you want to play Warcraft on it. You do have to get familiar with Linux to use this, though.
The Arduino's another good (possibly better) option. You can program it in C/C++/Java (and some other languages if you use some third party tools). It is low on RAM, though--you sometimes need to chain two Arduinos in a master-slave configuration to get stuff done.
The only issue is, you need to make your own sensors for the Raspberry Pi (sensors for the Arduino exist and some can be found here. In addition, there is some examples using NXT equipment here.) Generally not that hard (for example, a light/dark sensor can be made with a photodiode and resistor), though this is a big shift from the NXT (which was plug-and-play). You'll have to get your own servos and learn how they are controlled, and lots of other little things. You will have to get your hands dirty with soldering control boards/etc, and you will have to know a decent amount of electronics.
Otherwise you can just continue using the NXT, there are all sorts of premade sensors out there (though custom making your own sensor isn't that easy).