Large scale reverse engineering?

[torne]

First, yes, I do know what I'm talking about. I'm an embedded software developer who works on a realtime ARM embedded OS for a day job. I've reversed quite a lot of stuff before. :)

I'm quite interested in reverse engineering some Pokémon game on a large scale - exposing and documenting the engine in as much detail as possible. The editors that the community has developed are very impressive, but actually modifying the engine behaviour is mostly out of their scope. It's a cute game and the engine that powers it is very nice indeed, but prodding the contents of data structures is a little, well, awkward for my tastes. :)

It'd be neater to rip the ROM into the pieces it's composed of (individual tables/maps/sprites/etc) and be able to reassemble it from those same components. Removes the requirement to 'fit stuff in', and isolates the game from the engine. Identifying the components of the engine would make it easier to fiddle with too.

I'm probably looking at the GBA games, as D/P have a lack of readily available reference material at present.. identifying code by working backwards from the data locations is considerably easier than trying to work forwards from the entry point. :)

I can't find much information about the actual engine code. Does anything exist? How were the existing editing tools developed.. observation under emulation?

Just, well, wondering if anyone might have more docs than I can find. More stuff to start with is always good!

 

[Supreme Dirt]

First, yes, I do know what I'm talking about. I'm an embedded software developer who works on a realtime ARM embedded OS for a day job. I've reversed quite a lot of stuff before. :)

I'm quite interested in reverse engineering some Pokémon game on a large scale - exposing and documenting the engine in as much detail as possible. The editors that the community has developed are very impressive, but actually modifying the engine behaviour is mostly out of their scope. It's a cute game and the engine that powers it is very nice indeed, but prodding the contents of data structures is a little, well, awkward for my tastes. :)

It'd be neater to rip the ROM into the pieces it's composed of (individual tables/maps/sprites/etc) and be able to reassemble it from those same components. Removes the requirement to 'fit stuff in', and isolates the game from the engine. Identifying the components of the engine would make it easier to fiddle with too.

I'm probably looking at the GBA games, as D/P have a lack of readily available reference material at present.. identifying code by working backwards from the data locations is considerably easier than trying to work forwards from the entry point. :)

I can't find much information about the actual engine code. Does anything exist? How were the existing editing tools developed.. observation under emulation?

Just, well, wondering if anyone might have more docs than I can find. More stuff to start with is always good!

I myself have attempted this many times. Can't help you, but I'll look around and see what I can find.

I do have an idea, though.

For the Macintosh, there is a program named ResEdit. Possibly, if we had the offsets for everything, we could use it to view every little thing in the game.

I do know, however, that serebii.net has had a certain amount of success figuring out some systems in the game (such as formulas and the like) so it might help to check their. But don't post anything about it on their forums, because it will get you banned from there.

 

[torne]

For the Macintosh, there is a program named ResEdit. Possibly, if we had the offsets for everything, we could use it to view every little thing in the game.

I do know, however, that serebii.net has had a certain amount of success figuring out some systems in the game (such as formulas and the like) so it might help to check their. But don't post anything about it on their forums, because it will get you banned from there.

Ah, I didn't mean offsets of data so much (a lot of that has been documented and editors implemented already) - I meant the actual game engine code. Individual functions, how global state is managed, etc, in order to do things like changing menus, adding entirely custom features, etc. The best documentation of data formats is the code that accesses them, after all :)

Splitting up all the data is just a means to writing more convenient tools, and to working backwards into the actual code. I've not put a lot of time into this yet, but for example: the offsets used by AdvanceMap in its ini file to locate the map/tile data are actually the literal pools for the functions that the game engine uses to look them up. So, by working backwards from the offsets it uses, you can find the address of the function, and can then search the entire rom for branches that point to that address. This gives you every place where map data is looked up. Each place can be disassembled and a vague/good guess as to what it's doing, then you can continue working backwards. You end up at, say, the top-level function responsible for rendering the current map, or for detecting collisions with scenery when moving, or for spawning field Pokémon. :)

And actually, it's amusing how little of the rom actually contains ARM code. By this point in the series their scripting system and asset management is pretty standardised and very little has to be done in the actual code, which is the way to make a profitable series really (you need less programmers that way, as artists/scripters/writers can work directly on the game using their own tools which are going to be probably quite like the hacking community's *grin*).

 

[D-Trogh]

Well.. You can ASM-program 'new features'..
When Mastermind_X is back from his trip I'll point him to this thread. ;)

Mastermind_X created a Day/Night system for Pokémon Einsturm, made the ShinyHack and newly discovered all of the wolrld-map data for Pokémon FireRed (& LeafGreen ?).

You should talk with M_X when he's back. Or, you could try to contacht pika/loadingNOW from POKEGUIDE.
He reversed a lot of things. And knows a lot of things.

Also, Kawa and his friends are working on a new project called 'OpenPoké'.
[GameMaker alike program for creating Pokémon games]
For this project Kawa and Cearn are reversing a lot of GameMechanics/Info.
You can try to contact them too.

Anyway, whenever I try to find something..
// With my realy basic ASM knowledge (I can't make my own new functions.. :P)
.. I just use VBA's Tools + VBA-SDL-H (=Debugger).


I hope this helped,
[D-Trogh]

 

[Ooka]

Well I have no idea what you guys are talking about but wouldn't you have to have a program that makes the data viewable through screens. I mean how would you know what you were editing unless you were running on an unlz gba type system?

 

[D-Trogh]

Well I have no idea what you guys are talking about but wouldn't you have to have a program that makes the data viewable through screens. I mean how would you know what you were editing unless you were running on an unlz gba type system?

l0l.. what are you doing here then.. :-\
Ever heard of a HEX Editor + Knowledge?
VBA's Memory viewer, dissasembler, ..?
Ever heard of VBA-SDL-H? You are playing while you're debugging the game..
Anyway, doesn't matter.. - -'

 

[torne]

Also, Kawa and his friends are working on a new project called 'OpenPoké'.
[GameMaker alike program for creating Pokémon games]
For this project Kawa and Cearn are reversing a lot of GameMechanics/Info.
You can try to contact them too.

Saw that, but it doesn't seem to be very far along.. ;)
That's more or less the level of capability you would get out of full game engine documentation, also.

Anyway, whenever I try to find something..
// With my realy basic ASM knowledge (I can't make my own new functions.. :P)
.. I just use VBA's Tools + VBA-SDL-H (=Debugger).

Heh. Those are kinda useful, but No$GBA is a much better debugger if you buy the developer version. Oh, and my work laptop's licence for IDA Pro Advanced comes in handy :)

 

[Supreme Dirt]

Well, if you do find a way to reverse engineer, I could help with it.

Sounds like a pretty awesome idea, btw.

 

[torne]

Well, if you do find a way to reverse engineer, I could help with it.

Sounds like a pretty awesome idea, btw.

Not wanting to be rude, but if you don't already know what the way to reverse engineer a large binary is, then you probably can't help directly ;)

 

[Cartmic]

Hi I don't program an OS, but I use an ARM based one daily known as RISC OS.

 

[Supreme Dirt]

Not wanting to be rude, but if you don't already know what the way to reverse engineer a large binary is, then you probably can't help directly ;)

Meh, I guessed as much. Good luck, though.

 

[Mikasa]

Firstly, this isn't the greatest rom hacking forum, you may want to try one of the following: (sorry it won't let me post proper hyperlinks)
filbboard.de (a german site, more pokemon focused, you must register to access the rom hacking forum, if you post in english you'll most likely get a response in english, tend to be helpful) and

board.acmlm.org (better all-round hackers, more super mario focused, tend to 'look down' on pokemon hacks, but that's mostly because pokemon hackers tend to be kids who've downloaded an editor for it and don't really know how roms/emulation/uprocs work, if you make the same post you've made here you'll show you're not and they may be helpful).

I don't know if there's much info out there on the inner workings GBA games, I know the guys at filbboard had a load of info on the GBC games but they were z80 powered so not your bag. I've only briefly looked at the GBA code, so i don't know much about it but it seems very hard to follow (probably because it's from compiled C code) whereas the code for the GBC games is very easy to follow. GBC games were generally all written in ASM to begin with.

And on the question of how tools were developed, I personally try and search for the game data, and then search backwards (find pointers to data structures) until I find the bit of code that loads in that data, then poke around from there. It's entirely possible to work the other way..

I'd be interested to hear how you get on!

 

[Superkid11]

Now I'd really like to see how this works out! This could make it more exciting if not easier. I'd love to see how this works out.

 

[torne]

Well, you'll have to be patient, as I'm moving house at the moment and I haven't had time to make much progress. :)

 

[Supreme Dirt]

Well, you'll have to be patient, as I'm moving house at the moment and I haven't had time to make much progress. :)

I'm making a slight amount of progress learning to reverse engineer binary.
Wow, it is ridiculously hard.
I might possibly be able to help at some point, but not anytime soon.

 

[torne]

I'm making a slight amount of progress learning to reverse engineer binary.
Wow, it is ridiculously hard.
I might possibly be able to help at some point, but not anytime soon.

What're you trying to reverse? A large, complex game engine like Pokémon is not a good starting point for learning, I would've said..

I'd suggest starting with some binary that you have section and symbol information for - there is plenty to be learned there before you get onto reversing raw dumps from ROM.

 

[Supreme Dirt]

I am reverse engineering a game I made myself (not a rom, an executable)

 

[natu133]

woah, so, the idea ive got from this thread is:
Some dude's boss wants him to Reverse The engineering on a large scale, i dont know what that means, but i think it has something to do with...defragmenting...a ROM, and that another guy is trying to help..okay...what exactly do you mean by Reversing the Engeeneering..cause i might know something about it, but know it under a different name...my freinds are always talking about programs and technological things, but they say them using words i can understand...

 

[spenceroone3]

I knowthis is probably not the thread... but can someone please explain what the hell ARM code is?