Calling all RomHackers - Pokélink integration

[Jezzabel]

Introduction

Hey RomHackers

I'm Jez, one of the owners & Core Developer of Pokélink. A streaming app for Pokémon streamers and Pokémon-related content creators alike.

I've been building this for nearly 2 years, and support is pretty good. But one thing we get asked by streamers using your ROM Hacks, is "when will you enable support for -Insert Rom Hack name here-".
But without an intricate knowledge of the ROM's internal structure and usage of memory, this is a long and tedious process consisting of months of playing each romhack and the following steps for each data point:

• dumping the RAM,
• performing an action
• dumping the RAM again
• Searching for changes
• Repeating the above, until we drill-down to a specific value (single binary value, byte, range of bytes, etc)
• Possible Decryption of Blocks/etc
• Reversing this into a usable value
• Testing for edge cases and outliers

Pokélink TL;DR

It reads the memory directly from raw HEX values, parses & processes it into something like this

Spoiler:

Then something like these for a steamer's layout:

Spoiler:

What is Pokélink

Pokélink is an app that helps streamers & Poké-tubers create beautiful viewer-engaging streams when they stream/record Pokémon game playthroughs.

It enables a bridged connection between your emulator (DeSmuME, VBA & Citra) and the Pokélink app; it then provides a set of browser sources that can be included within OBS/SLOBS/etc. (We also allow people to manually control your party for non-emulated games)

Visual Demo:

Link to a preview of some of the themes

Currently Supported Games

• Stable Automated support for Vanilla & Randomized Gen 1 games (R/B/Y) (Missing: encounter location)
• Unstable (Not published) Automated support for Gen2 games (G/S/C) (Working on memory bank usage)
• Full Automated support for Vanilla & Randomized Gen 3 games (R/S/E, FR/LG)
• Stable Automated support for Vanilla & Randomized Gen 4 games (D/Pl/Pt - HP doesn't update In battle) (HG/SS - Full Support)
• Stable Automated support for Vanilla & Randomized Gen 5 games (B/W/B2/W2 - HP doesn't update In battle for all games)
• Full Automated support for Vanilla & Randomized Gen 6 games (X/Y/OR/AS - 2 known issues traced back to data structs mismatching between versions of the ROM)
• Full Automated support for Vanilla & Randomized Gen 7 3DS games (Sun/Moon/US/UM)
• No roadmap planned for Gen 7 Switch games (LGP/LGE - Waiting for Emulators to improve performance, monitoring them closely and in communication with the teams)
• No roadmap planned for Gen 8 games (Sw/Sh - Waiting for Emulators to improve performance, monitoring them closely and in communication with the teams - But we did have a PoC custom homebrew app that we got working, but it would have required a lot of R&D with adoption likely looking low)

Features

• Auto rom-detection
• Dynamic config generation and emulator setup
• Manual Badge updating
• Multi-player functionality allowing people to share their live team updates with each other in real time allowing all to show the status of all participants
• Automatic Route tracking for Nuzlockes (Of pokemon that have passed through the players party, routes can also be updated manually)
• A collection of Real time, browser sources (ready to go) that display the team and changes from the emulator in real time, to a web source that handles the graphical representation of that data.
• More is on it's way!

As you can see, there's a lot of work put into it, so to be able to branch out and include popular rom hacks would be awesome.

How you can help us, help streamers using your game

If you'd like to help us integrate with your ROMHack, then get in touch or post below. It's easy to integrate (ish) when we know the memory locations and data structures.

I don't know how much or little knowledge of these you'd be aware of or have explicit knowledge of, but I'd love to work with you and help streamers show off your game to the masses.

I want to open this discussion and see what you think, because the creators I see in the Pokélink community who use your games are begging for this integration.

I look forward to seeing in the replies, what you all think and any progress we can make.
Much Love
<3
- Jez

Further Info

• Join the community on Discord - https://discord.gg/MxRHFG2
• Support us on Patreon - https://www.patreon.com/pokelink
• Check out more info about the app on Youtube: https://youtube.com/playlist?list=PLAFYOBP5WK5UAVDg7YXPXzXTnBeSc5uWv
• Check out the latest news about the project on Twitter: https://twitter.com/PokelinkApp

​

 

[Sheep]

I'm not personally invested in fan games enough to make use of such a tool, but this sounds quite useful and I'm appreciative you guys are contributing so much to the fangame community! Thanks and good luck with it =)

 

[Jezzabel]

I'm not personally invested in fan games enough to make use of such a tool, but this sounds quite useful and I'm appreciative you guys are contributing so much to the fangame community! Thanks and good luck with it =)

Fair enough - We support all main-line Pokémon console based games too - updated a list in the post.

But this is more of me reaching out to the authors of RomHacks and working with us so we can improve streams, promote the games and just generally benefit the entire pokemon community surrounding playthrough-related content creation. <3

 

[jojobear13]

I'm the author of Shin Pokemon. It's a Gen 1 rom hack series primarily for fixing up the huge amount of bugs and oversights in pokemon Red and Blue, and it's probably the most bugfixed versions of these games that currently exist. That said, this (rather arduous) task has required me to become very familiar with the pokemon data structures and the wram and hram values in general for gen 1 games. Drop me a line with any questions if I can be of help to you on gen 1 stuff.

As for supporting Shin Pokemon, it is probably already supported in as far as the vanilla games since I didn't make any drastic changes to the memory or change the pokemon data structures. Only add-ons and reintegration of unused addresses in order to implement fixes and features. Though I guess if you had some ideas you could do something with the color engine, sound engine, trainer team moves, and other stuff that's been backported from Yellow version or is part of the expanded trainer AI.

 

[masterquestmq]

Hello,

My game is still in Beta but this looks very interesting. I'd totally be interested in getting involved in this

 

[Jezzabel]

I'm the author of Shin Pokemon. It's a Gen 1 rom hack series primarily for fixing up the huge amount of bugs and oversights in pokemon Red and Blue, and it's probably the most bugfixed versions of these games that currently exist. That said, this (rather arduous) task has required me to become very familiar with the pokemon data structures and the wram and hram values in general for gen 1 games. Drop me a line with any questions if I can be of help to you on gen 1 stuff.

OH GOD I HAVE SO MANY haha. It's cool that you're a gen1 specialist. That could come in SUPER Helpful! Sounds like you put a lot of work into that.

As for supporting Shin Pokemon, it is probably already supported in as far as the vanilla games since I didn't make any drastic changes to the memory or change the pokemon data structures. Only add-ons and reintegration of unused addresses in order to implement fixes and features. Though I guess if you had some ideas you could do something with the color engine, sound engine, trainer team moves, and other stuff that's been backported from Yellow version or is part of the expanded trainer AI.

• I want to be able to do identify what game is running uniquely (We have specific game ids that we try and match to the game ID from the game ID within the rom's header (first 350 bytes of the roms will contain the unique id like POKEMON_ BLUE, POKEMON_YELL and POKEMON_RED for example, Any ideas on the general landscape of Gen1 rom hacks?
  
  
• Currently in gen 1 (using vanilla memory locations (as far as I can tell) there's a few things that.. escape me/aren't there in memory, they might be in the same?)
  Gen 1 Data I get out currently: pid, species, speciesName, nickname, is_female, exp, level, hp, evs, move1, move2, move3, move4, status I'd love the locationMet out so we can add auto-route tracking for nuzlockes for all of Gen 1.
  
  
• I think I have badge auto-tracking implemented in Gen1, but it won't be implemented in the app due to no other game supporting badges. (If you have knowledge of badges in any other games, that'd be awesome. (I believe in most games, it's not technically a `badge obtained` address; It's more of a progression marker? It's been a long time

 

[jojobear13]

I want to be able to do identify what game is running uniquely (We have specific game ids that we try and match to the game ID from the game ID within the rom's header (first 350 bytes of the roms will contain the unique id like POKEMON_ BLUE, POKEMON_YELL and POKEMON_RED for example, Any ideas on the general landscape of Gen1 rom hacks?

Not many bother with changing the rom header. This is something the authors need to do themselves.

Currently in gen 1 (using vanilla memory locations (as far as I can tell) there's a few things that.. escape me/aren't there in memory, they might be in the same?)
Gen 1 Data I get out currently: pid, species, speciesName, nickname, is_female, exp, level, hp, evs, move1, move2, move3, move4, status I'd love the locationMet out so we can add auto-route tracking for nuzlockes for all of Gen 1.

Gen 1 does not have Caught Data as part of the pokemon data structure. This was added in gen 2. Trading a pokemon back and forth between gen 1 and gen 2 will erase this data.

In fact, the only gen 2 addition to the pokemon data structure that gets preserved is the Held Item byte. That's because it occupies the same space as Catch Rate in the gen 1 data structure (which did nothing for a caught pokemon). This is why a wild kadabra always holds a twisted spoon when traded into gen 2; the value for twisted spoon was purposefully set to its gen 1 catch rate.

Gen 1 does not have gender or shiny values. However, these are deduced in gen 2 by using DVs (precursor to IVs used in the first two gens). Gender is determined by comparing the attack DV to the gender ratio value of the species using a lookup table. A pokemon is shiny if the 16 bits of its DV values make "xx1x 1010 1010 1010" wherein "x" is a don't-care. Naturally, gender and shininess can be backported into Gen 1 since only the DV values actually matter.

This is the party pokemon structure for Gen 1.

\1Species::    db	;+$00
\1HP::         dw	;+$01
\1BoxLevel::   db	;+$03
\1Status::     db	;+$04
\1Type::
\1Type1::      db	;+$05
\1Type2::      db	;+$06
\1CatchRate::  db	;+$07
\1Moves::      ds 4	;+$08
\1OTID::       dw	;+$0C
\1Exp::        ds 3	;+$0E
\1HPExp::      dw	;+$11
\1AttackExp::  dw	;+$13
\1DefenseExp:: dw	;+$15
\1SpeedExp::   dw	;+$17
\1SpecialExp:: dw	;+$19
\1DVs::        ds 2	;+$1B
\1PP::         ds 4	;+$1D
\1Level::      db	;+$21
\1Stats::
\1MaxHP::      dw	;+$22
\1Attack::     dw	;+$24
\1Defense::    dw	;+$26
\1Speed::      dw	;+$28
\1Special::    dw	;+$2A

Note that EVs do not exist. Gen 1 and 2 have a word of space for each stat in which to track stat experience.

Also, DVs are stored in only 2 bytes with each stat using 1 nibble in the following order: attack, def, speed, special. Where is HP? it's calculated from the other four DVs. The HP DV is increased for each of the other DVs that are odd-numbered.
Attack = Odd, HP += 8
Defense = Odd, HP += 4
Speed = Odd, HP += 2
Special = Odd, HP += 1

Status condition uses a single byte. A bit being set indicates having the corresponding condition. Bit 7 is unused. Bit 6 is paralysis. Bit 5 is frozen. Bit 4 is burned. Bit 3 is poisoned. Bits 2 through 0 are the sleep counter wherein the 'mon is asleep if this is above zero (giving a possible 1 to 7 turns of sleep) .

The party data ram addresses are as follows for Red version.

wPartyCount::   ds 1 ; d163
wPartySpecies:: ds 6; d164
wPartyEnd::     ds 1 ; d16a
wPartyMons::
wPartyMon1:: party_struct wPartyMon1 ; d16b
wPartyMon2:: party_struct wPartyMon2 ; d197
wPartyMon3:: party_struct wPartyMon3 ; d1c3
wPartyMon4:: party_struct wPartyMon4 ; d1ef
wPartyMon5:: party_struct wPartyMon5 ; d21b
wPartyMon6:: party_struct wPartyMon6 ; d247

wPartyMonOT::    ds 66 ; d273
wPartyMonNicks:: ds 66 ; d2b5
wPartyMon1:: party_struct wPartyMon1 ; d16b
wPartyMon2:: party_struct wPartyMon2 ; d197
wPartyMon3:: party_struct wPartyMon3 ; d1c3
wPartyMon4:: party_struct wPartyMon4 ; d1ef
wPartyMon5:: party_struct wPartyMon5 ; d21b
wPartyMon6:: party_struct wPartyMon6 ; d247

Note that text names are terminated with the $50 hex value.

Also, Blue and Yellow are at a slightly different offset with wPartyCount starting at the following addresses:

Blue: AF2C
Yellow: 2F2C

But most hacks are based on pokered disassembly and so stick with d163 as the starting address.

This is just the party data for when 'mons are not currently out or when you are running around outside of battle. There is an entirely different battle structure just for active pokemon and 3 extra bytes for both player and enemy for tracking extra effects (like toxic counter, trapping moves, transformed, etc). Check out the pokered disassembly and start poking around wram.asm to see what all sorts of cool stuff you can find.

I think I have badge auto-tracking implemented in Gen1, but it won't be implemented in the app due to no other game supporting badges. (If you have knowledge of badges in any other games, that'd be awesome. (I believe in most games, it's not technically a `badge obtained` address; It's more of a progression marker? It's been a long time

Not that I know of. But the BadgeObtained address is just a flag array. All event markers in most pokemon games are just flag arrays of consecutive bytes.

 

[Meister_anon~Master_o f_None]

Fair enough - We support all main-line Pokémon console based games too - updated a list in the post.

But this is more of me reaching out to the authors of RomHacks and working with us so we can improve streams, promote the games and just generally benefit the entire pokemon community surrounding playthrough-related content creation. <3

If you don't already you should setup to work with mgba, its the best gba emulator to date, the others are pretty out of date, also the demo youtube vid is down. Least when I tried to click it.

 

[Jezzabel]

If you don't already you should setup to work with mgba, its the best gba emulator to date, the others are pretty out of date, also the demo youtube vid is down. Least when I tried to click it.

Working with mGBA means adding another layer of interaction, adding the functions that mGBA offers, and then testing them all, it is on our list, but it's not our #1 priority currently.

The link is valid, but the embed isn't working for some reason - I've updated the original post with a link to the video - https://youtu.be/vgocjtnG_eM?t=47

 

[Meister_anon~Master_o f_None]

Working with mGBA means adding another layer of interaction, adding the functions that mGBA offers, and then testing them all, it is on our list, but it's not our #1 priority currently.

The link is valid, but the embed isn't working for some reason - I've updated the original post with a link to the video - https://youtu.be/vgocjtnG_eM?t=47

gotcha, and I figured out the embed situation a little while ago myself, its kind of weird and unintuitive, instead of putting the entire url in the embed brackets what youhave to do, is only add the end section with the letters of the specific youtube vid. this part vgocjtnG_eM?t=47
here just hit respond on my comment and copy out what I have below to edit your post. *edit watching the vid really helped with understanding what this is, its pretty cool.*

 

[Jezzabel]

gotcha, and I figured out the embed situation a little while ago myself, its kind of weird and unintuitive, instead of putting the entire url in the embed brackets what youhave to do, is only add the end section with the letters of the specific youtube vid. this part vgocjtnG_eM?t=47
here just hit respond on my comment and copy out what I have below to edit your post. *edit watching the vid really helped with understanding what this is, its pretty cool.*

Oh sweet, thanks man, done!

 

[Meister_anon~Master_o f_None]

Oh sweet, thanks man, done!

No prob buddy :)

 

[Jezzabel]

If you don't already you should setup to work with mgba, its the best gba emulator to date, the others are pretty out of date, also the demo youtube vid is down. Least when I tried to click it.

Just an update - I've now spoken to the developer behind mGBA and they've not added lua support yet, so we cannot add support for it. But I've supported their patreon and hopefully it can be added soon.

 

[Meister_anon~Master_o f_None]

Just an update - I've now spoken to the developer behind mGBA and they've not added lua support yet, so we cannot add support for it. But I've supported their patreon and hopefully it can be added soon.

oh ok, and cool I appreciate you taking the time to update me on that.

Thank-you.

 

[Jezzabel]

RPG Maker Updates!!

I've had a chat with somebody recently who wanted to ask about RPG maker's pokemon base

if this would help pokemon essentails v19 is open source on Github rn im on 18.1 but am looking into updating

By the looks of it, It's possible for a few scripts to be written for pokemon essentials to hook into the events where something changes and output a file in the same format we support pretty easily.

(By all means --- I mean .. if somebody wants to add support for this xD)

Ideally it wouldn't be writing a file due to unnecessary writes to the users' drive. Instead using a RPC or Web Socket server. But :Scraggshrug: options!
I think I can pretty confidently say that myself, Linky or @Nimbus won't be working on this due to lack of demand and the amount of things we have to do in the current backlog :stuck_out_tongue:

But I'm more than happy to get it integrated, if somebody builds it, once it's stable..

 

[Jezzabel]

Massive shoutouts so Soupercell, a private pre-release of Pokélink now has support for 70% of radical red.

It's currently actively with Patrons + alpha testers and will be dropping to the public in the near future

 

[hjk321]

Decomp hacks are going to be difficult to support for Pokélink. Every single pointer you're trying to read from will be shifted on every update/build, which is going to massively increase the maintenance needed for these hacks on your end.. You essentially have to work very closely with decomp authors every time they update, which will probably be taxing on both sides.

On the other hand, if the hack authors care enough, you can get them to send you their .map file of their official build, which is basically a giant table of pointers and their labels, both in ROM and RAM. All you need to do is look at the map file in a text editor and find the variable names of what you're looking for. (i.e. "gPlayerParty" or "gBattleMons"). You might also need a sym dump to get the rest of the variable pointers, but you'll need to ask your local neighborhood decomp hacker to provide that as well.

Where it gets complicated is that since decomp offers more flexibility in modifying fundamental elements of the game, you'll probably have to code in custom support for each individual hack for the data structures you're trying to read, since they've been changed from vanilla. For example, in my decomp hack (and a few others I am aware of) the BoxPokemon structure has been changed from vanilla, to the point where the pokemon encryption was completely removed, all the data were shifted around into bitfields, and it takes up 64 bytes per Pokémon instead of 80. This size difference, in turn, snowballs onto every single other structure in the game that has to do with a Pokémon - party data, pc boxes, hall of fame, daycare... and that's just with a single change. I'm not quite sure exactly what information you're trying to grab from memory but I can imagine the integration you're trying to get won't be the same as vanilla in most total conversion decomp hacks.

So I guess what I'm saying is that you will need massive support from the hack author's side for decomp hacking, and not only that, but *continued* support with every update. I would recommend trying to convince them that it's in their best interest to help Pokelink support their hack because it will get them more players or whatever. I could imagine that many devs wouldn't be too keen on the additional workload. I might personally be interested in the distant future but my hack isn't very close to release.

I can answer some of your decomp questions if you want.

 

[Pokabbie]

Hiya! I'm the dev for Emerald Rogue and I've had a couple of people mention getting support for this.
As someone pointed out above this is something that's gonna be difficult to do for decomps as these locations will likely change from patch to patch, so I was wondering if we could discuss a way to keep compatibility going forward?

Perhaps a more generic API for decomp hackers to expose the head pointer for the party in sav data and/or then some config we could provide for matching species and move ids?
In my case I built off RH-Hideout's Emerald Expansion project so any config could be re-used from any other project.

 

[mgriffin]

Perhaps a more generic API for decomp hackers to expose the head pointer for the party in sav data and/or then some config we could provide for matching species and move ids?

The GF ROM header has the offsets for party data and species/move names (among other things). wrt the party data, the big problem would be if the Pokémon structure was different, or if somebody changed the save sector structures (e.g. by making the footers smaller).

If you need more data perhaps it would be possible to create a branch/document that describes an extended GF ROM header.