Why Does The Emulator Have To Load The Whole Rom?, Still pondering this 64MB thing... Options

[GaijinPunch]

As the title says - just wondering why an emulator has to load an entire ROM in one go? Is it impossible to only load the parts of the ROM it needs at specific times? I assume it would take a lot of game-specific changes for the driver, but is it even possible?

Hmm.....

[noodle1009]

the main problem right now is the fact that mameox is is a direct port of mame with some enhancements. Mame loads the entire rom into memory by design - since rom contents are in memory to begin with, this is closest to the actual workings of an arcade machine. The goal of mame is not speed, not memory savings, not functionality, rather the most accurate emulation of arcade hardware possible in software.

You can look at the drivers in the source and see how they're written. You can see how when they load they allocate the memory that they need. You can also see that there's a crapload of them .

If you really want to see an eye opener, get the newest mame32 and fire it up. Take a problem rom that's out of memory and boot it up. Look in windows task manager and see how much memory it has allocated.

Pretty impressive, huh?

Erik was very smart and actually made mameox unload the specific drivers that it wouldn't need prior to loading a rom. This enables mameox to free up on average close to 54 MB for all roms prior to the loading sequence. If this was a direct port with no enhancements, many more roms would not load at all.

Again, at some point it may be possible to put in some driver hacks that allow some games to load, but right now we're working to try to get everything in the UI we want in, lightgun support working correctly, and all of the roms that have memory to load with rock solid operation. Once eabair and opcode starts running out of easy things to do, maybe we'll see some investigation into virtual memory to even see if it's feasible. Until then, they'll try to milk every last byte out of the RAM they can.

This post has been edited by noodle1009: Jul 8 2003, 03:05 AM

[GaijinPunch]

Sounds like a good answer to me.
Thanks - very informative!

[feflicker]

To add yet another dimension,

When was the last time you were able to run 1/2 of an exe file? It just doesn't work that way... Remember, this is all machine code at some level...

It is an interesting question, however.

[desertboy]

I always thought of it like this

Probably doesn't apply to mame roms though

if you have to pay per megabyte for storage (EG N64) as oppossed to flat rate per 700meg (PSX/Saturn) then you use your other advantages (very quick data access) to try and compensate.

Take something like Zelda64 it only has a couple of models of Zelda on the rom whenever Zelda's model is loaded it's load from one specific place in the rom and samples it might play somewhere else in the rom. Due to instant access nature of roms, the only thing fast enough to store the rom in is ram. Everytime something is accessed on the HD the perfomance hit would be huge (You'd have to write part of the contents of ram to the HD before you can load in the needed data) because CD's stream (A lot slower than you HD) you don't have this problem as it's always read sequentially a lot less random accessing of the "Rom".

I'm sorry if this post seems confused but I think I might be pretty confused at the moment.

[noodle1009]

QUOTE (desertboy @ Jul 8 2003, 08:40 AM)

I always thought of it like this Probably doesn't apply to mame roms though if you have to pay per megabyte for storage (EG N64) as oppossed to flat rate per 700meg (PSX/Saturn) then you use your other advantages (very quick data access) to try and compensate. Take something like Zelda64 it only has a couple of models of Zelda on the rom whenever Zelda's model is loaded it's load from one specific place in the rom and samples it might play somewhere else in the rom. Due to instant access nature of roms, the only thing fast enough to store the rom in is ram. Everytime something is accessed on the HD the perfomance hit would be huge (You'd have to write part of the contents of ram to the HD before you can load in the needed data) because CD's stream (A lot slower than you HD) you don't have this problem as it's always read sequentially a lot less random accessing of the "Rom". I'm sorry if this post seems confused but I think I might be pretty confused at the moment.

These are valid points. For the record CDs do stream and game systems are made to take advantage of this. Your PSX does not load an entire game into memory, rather, only what it needs. But this console was let up to work that way, arcade games were not (unless you're talking about laserdisc ones, but that's different ).

People need to understand that to load these larger roms, you're going to have to rewrite parts of mame to do it. MAMEoX was a port, not someone's attempt at reinventing MAME.

This post has been edited by noodle1009: Jul 8 2003, 01:20 PM

[lowry383]

I wonder if there's a way to take a portion of the xbox hd and convert it to some sort of ram for the software emulators like the n64 and mame......

[noodle1009]

QUOTE (lowry383 @ Jul 9 2003, 06:51 AM)

I wonder if there's a way to take a portion of the xbox hd and convert it to some sort of ram for the software emulators like the n64 and mame......

from the MAMEoX faq (addressing 'virtual memory', or 'HD RAM')

QUOTE

Q.  Everyone says creating a swap file for MAMEoX to use would solve the      problem of running larger roms in MAMEoX!  Windows uses a swap file!       It must be really easy to implement!  Why don't you guys use      virtual memory in MAMEoX?  It's probably easy to do!  What's the      holdup?  By the way, I am not a coder and I have never looked at      code in my entire life. A.  First, while this is something that the coders might decide to tackle      in the far-flung future (and we're talking _far_ flung here) it's      not likely this will work very well with MAME for a couple of reasons.           MAME currently loads roms into RAM, so it can emulate games properly.      Rom information that is dumped into your rom file was originally      present in memory to allow the arcade machine quick access to its      contents.  Since the files were in memory, access to this information      was extremely quick from an I/O perspective (I/O is input/output      for the non-technical).  I/O to a hard drive, which is what would be      used in MAMEoX, because of lack of RAM to use, would be a _lot_      slower than memory I/O wait.  It's not even clear at this time if      the increased I/O time would work well or at all with roms loaded      in MAMEoX.  To make it work well (if at all), the MAME core would      need to be investigated so we would know the proper things in each      rom to send to virtual memory.  It's likely the source code of      MAME would need to be 'hacked' to accomplish this (which is      something the coders are not excited about doing, MAME is already      hacked enough by itself to begin with :) ).      This is beside the point that any sort of virtual memory system for      MAMEoX to use would have to be written from scratch.  There is no      'virtual memory' support that can be ported from the MAME core, and      MAMEoX is a port of MAME to the X-Box that is optimized to run as      many games as possible on your X-Box.  The developers, if they      decide to investigate this in the future, will more than likely only      do so after they have milked every last byte out of the X-Box's RAM.      If you could not tell by the tone of the question, the MAMEoX team      is not fond of receiving 'demands' from it's users for this,      especially when the tone of the question would have anyone believe      adding this sort of 'feature' is simple.      In closing, MAMEoX is open source.  If you think that implementing      virtual memory is easy, you're more than welcome to check the source      out of CVS and do the work for us. :)  Isn't the open source model      wonderful?

This post has been edited by noodle1009: Jul 9 2003, 05:00 AM

[lowry383]

ok thanx for the post. That definetly answers my question. Btw, thanx for not flaming me for not reading the mamox faq.

[ejx982]

I remember back when Dragon Warrior 4 (NES) was dumped, they broke it up into 4 parts to allow the first emulators to play it, before NES Emulators were able to load 1MB ROMS. I may be slighly mistaken however, as the total ROM may have been stored into 4 smaller parts on the original cart.

I always though you could PLAY split roms, but you may never be able to finish it. (ie, how do you jump from the 1st 1/2 to the 2nd)

ejx982

[noodle1009]

QUOTE (ejx982 @ Jul 9 2003, 03:49 PM)

I remember back when Dragon Warrior 4 (NES) was dumped, they broke it up into 4 parts to allow the first emulators to play it, before NES Emulators were able to load 1MB ROMS. I may be slighly mistaken however, as the total ROM may have been stored into 4 smaller parts on the original cart. I always though you could PLAY split roms, but you may never be able to finish it. (ie, how do you jump from the 1st 1/2 to the 2nd) ejx982

MAME loads the entire rom into memory when it loads the driver. Without changing the MAME source, or hacking it, this is just the way it's going to be for now.

It's suprising at first to see how much memory some 'smaller' games require. For example, Gunbird 2 isn't that big and at first glance would seem to be able to run. However, load it up in MAME32, then check your task manager. Gunbird 2 allocates over 210,000 K of memory! (That's close to 200 MB).

A lot of these drivers when they initialize are set up to allocate as much memory as the original arcade cabinet had. Again, the goal of mame is machine perfect emulation. MAMEoX is not attempting to rewrite the goals MAME is trying to achieve.

[GaijinPunch]

Oh, you bastard.
210MB? I wanted to play Gunbird 2.

[noodle1009]

QUOTE (GaijinPunch @ Jul 9 2003, 04:29 PM)

Oh, you bastard. 210MB? I wanted to play Gunbird 2.

yeah, me too

Luckily I have it for my DC. I know it's not the same, but it's a good port.

[woo]

I think we all have to acknowledge the work that has already been done on MAMEoX. I really do like the new frontend. It would be nice to see better support for artwork, etc.. in the future, but it's very nice and clean as it is. Most impressively, for me, is the amount of memory that has been squeezed out for us by unloading unneccesary drivers. Fantastic work!

I'm a big MAME fan (full set collector with mame testdrivers) and MAME on XBOX was always a hope of mine. I didn't realise before (never had any reason to research) that MAME loaded the complete rom into memory. That's dissapointing to hear, because even if there was an open source way of using the HDD for virtual memory, it wouldn't help MAMEoX, without a complete rewrite of the MAME core itself. Still.. always grateful for being able to play those classics on my XBOX. It somehow has the edge over playing it on my PC (even though my PC has an XBOX controller hooked up to it).

Great work Noodle1009, team and all contributers.

on a side note: Is it just me, or does anyone actually think the MAME frontends out there (on PC) are a bit, well... crap? heh. I've seen a virtual arcade one (a 3D modelled arcade) but, it's limited. I'd love to see something like this on XBOX, though I appreciate that's unlikely to happen.

[noodle1009]

QUOTE (woo @ Jul 9 2003, 04:52 PM)

I think we all have to acknowledge the work that has already been done on MAMEoX. I really do like the new frontend. It would be nice to see better support for artwork, etc.. in the future, but it's very nice and clean as it is. Most impressively, for me, is the amount of memory that has been squeezed out for us by unloading unneccesary drivers. Fantastic work! I'm a big MAME fan (full set collector with mame testdrivers) and MAME on XBOX was always a hope of mine. I didn't realise before (never had any reason to research) that MAME loaded the complete rom into memory. That's dissapointing to hear, because even if there was an open source way of using the HDD for virtual memory, it wouldn't help MAMEoX, without a complete rewrite of the MAME core itself. Still.. always grateful for being able to play those classics on my XBOX. It somehow has the edge over playing it on my PC (even though my PC has an XBOX controller hooked up to it). Great work Noodle1009, team and all contributers. on a side note: Is it just me, or does anyone actually think the MAME frontends out there (on PC) are a bit, well... crap? heh. I've seen a virtual arcade one (a 3D modelled arcade) but, it's limited. I'd love to see something like this on XBOX, though I appreciate that's unlikely to happen.

I will forward your compliments to the coders I'm just a tester and a MAMEoX fan, nothing more.

Working on the UI is easy in comparison to doing something like virtual memory.

If you have some specific requests for enhancements to the UI, we would love to hear them. There is an RFE (requests for enhancements) section on the sourceforge page, please put them there. We have an excellent couple of artists that has been throwing together some graphical stuff that we've been able to put into the UI - this is the reason it looks great IMHO today! Screenshot support in the UI is going to go in at some point (I am _really_ looking forward to that)!

Some ideas might not be feasible (we ran into a problem with history.dat, for example) but I think the UI has a lot of potential - with your input. So let us know what you'd like to see in MAMEoX, the developers are really interested in hearing your ideas (but get to requests as their time allows, so be patient - very patient)!