The legacy of Nintendo’s Game Boy may not end with the 3DS, but two industry analysts suggest the Kyoto, Japan-based company’s line of portable video game players and its ilk may be past their prime.

Nintendo’s 3DS and Sony’s upcoming PlayStation Vita are facing increased pressure from smartphones and other game-capable mobile devices for both consumers’ time and money. While the handhelds are targeting two different audiences – the Vita and 3DS are intended for the traditional video game crowd whereas gaming tends to be a secondary function for the typical mobile device user – they are ultimately after the same mind share.

“The market for adult casual gaming on portable gaming hardware has practically disappeared since … smartphones and tablets have arrived,” explains Peter Warman, CEO of video game research firm Newzoo.

However, he is quick to note that if demand for dedicated gaming hardware exists, there is still room for portable handsets such as the 3DS and Vita, adding: “As long as Apple and Samsung do not promote their iPod-type devices as portable game consoles for kids, the 3DS might be around for a while … [and] the Vita will cater perfectly to [tech-minded youngsters].”

Although the market for dedicated video game handhelds is there, it is shrinking. Sony and Nintendo will be able to coexist in the space, but the era of dedicated gaming portables selling hundreds of millions of units could be a thing of the past.

“The sandbox may not be as big as it was back in 2009, but there is still plenty of room for Sony and Nintendo to generate significant profits, as well as the third-party publishers that support them,” Jesse Divnich, vice president of capital research and communications at Electronic Entertainment Design and Research.

“There will always be a large and significant market for dedicated portable gaming, but we do believe it is a shrinking category that will eventually plateau to a sustainable market size for at least two competitors,” he asserts.

As smartphones, tablets and similar devices become more affordable, they are accounting for greater portions of mobile software revenue. In 2009, Apple’s iOS and Google’s Android platforms made up only 19 percent of total U.S. portable game software revenue, according to data from the NPD Group and mobile analytics firm Flurry. As of 2011, they own 58 percent of the market.

Conversely, Sony has lost 5 percent market share and Nintendo is down 34 percent in the same two-year period. While Nintendo seems content with its position in the market, Sony appears to be positioning its Vita as a hybrid platform between dedicated video game players and casual mobile devices. Apps for social sites such as Facebook and Twitter are in development for Vita, while Sony announced the PlayStation Suite SDK as well, which enables Android developers to create Vita-compatible games.

Earlier this year, Sony also launched the Sony Ericsson Xperia Play, an Android-powered smartphone that features a full PlayStation control pad and the ability to play specific games from the video game maker’s extensive back catalog of PlayStation and PSP software.