path: root/docs/notes.md

# research

## DS / DSi emulators

wanted features:

- **DS and DSi version emulation (DSi for later)**
  
  DS version of pictochat can still receive images with rainbow drawings, but
  doesn't support sending
- **firmware booting**
  
  the phat DS and DS lite's firmware had pictochat built-in instead of as a
  standalone app like on the DSi. to launch pictochat, firmware booting must be
  possible.
- **local multiplayer network emulation**
  
  required for testing sending and receiving
- **local multiplayer packet capture**
  
  required for reverse engineering the packet format
- **local multiplayer packet injection**
  
  required for testing forged packets
- **open source**
  
  DIY packet injection anyone?
- **linux-compatible**
  
  just a preference
- **debugging (ideally using GDB or similar interface)**
  
  if all else fails, reading and/or injecting packets into memory using a
  debugger might be the only option for testing forged packets. a debugger
  could also help with reverse engineering the decoding process of a pictochat
  packet/message

### [DeSmuME][desmume]

- phat ds only
- no firmware booting
- no local multiplayer emulation
+ open source
+ cross-platform
- no debugger

DeSmuME is focused on high quality/performance emulation for single player
titles, and actively rejects requests for WiFi emulation.

### [melonDS][melonds]

+ ds and (experimental) dsi support
+ firmware booting support
+ local multiplayer emulation
+ allows patched-in or real network packet capture using loopback network
  adapter and wireshark
+ loopback network adapter also (probably) allows injecting forged packets
+ open source
+ cross-platform
- no debugger

melonDS is a very new emulator that is still actively being developed. melonDS
is focused on accurate emulation and is the only one on this list with
sufficiently advanced local multiplayer emulation.

### [no$gba][nogba]

+ supports DS and DSi games
+ firmware booting support
- no local multiplayer emulation
- closed source
- windows only
+ very advanced debugger

[toolchaingenericds]: https://bitbucket.org/Coto88/toolchaingenericds/src/master/
[nogba]: https://problemkaputt.de/gba.htm
[melonds]: https://melonds.kuribo64.net/
[desmume]: https://desmume.org/

## MelonDS hacking

source: <https://git.pipeframe.xyz/fork/melonDS>

[masscat-nifi]: <https://web.archive.org/web/20090202194241/http://masscat.afraid.org/ninds/proto_info.php>

- melonDS @ Config > Wifi settings "Local multiplayer features do not use the same network protocols as online play"
- comment @ src/Wifi.cpp:46 "multiplayer host TX sequence"
- references to `RFTransfer_Type{2,3}` @ <https://www.problemkaputt.de/gbatek.htm#dswifirfchip>
- nintendo ds ni-fi protocol @ [masscat-nifi]
- melonDS emulates actual 802.11b frames
- the protocol does not appear to be encrypted:

  ![](../assets/ws-no-encrypt.png)

  the string `lork` is visible as plain text in the hexdump (offset 0x0056),
  which appears to be some kind of 16-bit encoding of the username set on the
  emulator used to capture these packets (this is likely UCS-2 and not UTF-16
  as suggested by [masscat-nifi] which specifies UCS-2).
- The messages are not sent as single packets. The Ni-Fi protocol appears to set
  up a constant stream, and messages are sent across multiple frames.
- A full height filled-in message results in 64 packets (with Wireshark filter
  `nifi.type.enum == 1 && pictochat && frame[0x1a] == 0x02 && frame.len ==
  246`)
- PictoChat does not appear to send messages when you are in a chat room by
  yourself, so local multiplayer / Ni-Fi emulation is required for capturing
  message content

### Message sizing/cropping

- PictoChat automatically crops messages to the smallest height (at the fixed
  intervals shown as notebook lines in the edit field on the bottom screen).
  Message content can be on the notebook line itself without causing the
  cropping algorithm to 'allocate' more space; the allocation only happens if
  any of the pixels on the line *below* the colored line are used. The method
  used to crop the messages also ensures that the username label in the top
  left does not obstruct or remove any content.

  ![](../assets/pictochat-msg-height-1-4.png)
  ![](../assets/pictochat-msg-height-5-draw.png)
- In addition to being cropped before sending, the message border appears to
  mask the drawing:

  ![](../assets/pictochat-msg-crop.png)
  ![](../assets/pictochat-msg-crop-lork2.png)

  The above image was obtained after the following steps:
  - Fill in the top row of the message with the black pen, ensuring the entire
    notebook line is colored in as well
  - Send message (displayed as top message)
  - Copy message
  - Erase small portion of black area on the right side (displayed as bottom
    message)
  
  Notable observations:
  - The message content has a 1 pixel border (padding/margin) on all sides
    (except the bottom and corners) that can not be filled in. The bottom
    border is correctly displayed for received messages, but the edit field has
    an additional row of pixels that directly touch the message border on the
    bottom which is not visible in the received version of the message (i.e. is
    being masked).
  - When any message is cloned, additional pixels that were masked on the top
    screen become visible again in the edit field. This includes the bottom row
    of pixels, as well as the two rows of pixels shown in the single line
    message picture.
  - The **apparent** drawable area (including obstructed top-left corner) is
    228x80 pixels.

Observed package counts (no resends) for messages of different sizes:
|draw area|display height|packet#|
|-|-|-|
|top-left pixel of row 1 only|1|13|
|row 1|1|13|
|row 2|2|25|
|row 3|2|25|
|rows 1-2|2|25|
|rows 1-3|3|38|
|rows 4-5|3|38|
|rows 2-3|3|38|
|rows 1-4|4|51|
|rows 1-5|5|64|
|rows 2-5|5|64|

Notable:
- Messages that draw below line 1 while keeping line 1 empty can't be cropped
  to utilize line 1 as the username label is in the way. The captured packet
  counts suggest that the content of line 1 is still sent (although empty).
- The messages are cropped *before* sending, as suggested by the packet counts
- The message content is masked when displayed in the message log on the top
  screen

### Message content

![](../assets/pictochat-msg-pattern.png)
![](../assets/ws-msg-pattern.png)

- All messages with interesting content have Ni-Fi header type 1 (CMD).
- PictoChat messages appear to be sent over frames of length 0xf6 (246)
  regardless of actual message size.
- All frames appear to be sent exactly 5 times. 'New' frames have a value of
  0xe004 at offset 0x0026, while resends have a value of 0xf000 instead.
- There are a lot of messages with length 0xaa (170), these appear to include a
  random 32-bit value at offset 0x0046.
- The message content appears to be sent unencrypted (patterns of 0x0 and 0x1
  nibbles clearly visible in hexdump).
- Offset 0x004d - 0x00ec appears to be used to send the actual message content:

  ![](../assets/pictochat-msg-crop-lork2.png)
  ![](../assets/ws-msg-fill-top.png)
  ![](../assets/ws-msg-fill-mid.png)
  ![](../assets/ws-msg-fill-bot.png)

  (excerpts from the start, middle and end frames of a completely filled
  message)
- The completely filled message also shows an interesting pattern at the end,
  hinting at a slightly odd image codec
- Message length **can not** be used to consistently check which packets
  contain PictoChat message data.

<!-- #### MAC addresses

The following table of addresses may be visible in the packet captures.

|address|label|source|
|-|-|-|
|`00:09:bf:11:22:33`|Default firmware MAC|[melonDS/src/SPI\_Firmware.h:37](../melonDS/src/SPI_Firmware.h)|
|`03:09:bf:00:00:00`|Multiplayer CMD|[melonDS/src/Wifi.cpp:42](../melonDS/src/Wifi.cpp)|
|`03:09:bf:00:00:10`|Multiplayer Reply|[melonDS/src/Wifi.cpp:43](../melonDS/src/Wifi.cpp)|
|`03:09:bf:00:00:03`|Multiplayer ACK|[melonDS/src/Wifi.cpp:44](../melonDS/src/Wifi.cpp)|
|`00:f0:77:77:77:77`|Access point|[melonDS/src/WifiAP.cpp:37](../melonDS/src/WifiAP.cpp)|
|`10:00:de:ad:be:ef`|Instance 1 (`lork`) firmware MAC|me|
|`20:00:de:ae:02:ff`|Instance 2 (`lork2`) firmware MAC|me|
|`ff:ff:ff:ff:ff:ff`|Broadcast||

In melonDS's settings, the firmware MAC addresses for `lork` and `lork2` are
set to `11:00:de:ad:be:ef` and `22:00:de:ad:be:ef` respectively, but these
always get mangled and sent as the addresses noted in the table above. I have
no idea why this happens. -->

#### Analysis

See [the wireshark dissector](../wireshark/pictochat.lua)

<!-- ![](../assets/ws-msg-fill-mid.png)

|offset|type|description|
|-|-|-|
|0x00|`u32`|Ni-Fi: Magic (0x4e494649)|
|0x04|`u32`|Ni-Fi: SenderID (melonDS InstanceID)|
|0x08|`u32`|Ni-Fi: Type|
|0x0c|`u32`|Ni-Fi: Length (after Ni-Fi header)|
|0x10|`u64`|Ni-Fi: Timestamp|
|
|0x18|`u16`|PictoChat: Message type??? (0)|
|0x1a|`u16`|PictoChat: Resend (2=New, 0=Resend)|< !-- I assume u16 because the next byte is always 0x00 -- >
|0x22|`u16`|PictoChat: Length (offset 36 of complete packet length)|
|0x28|`u8[6]`|PictoChat: multiplayer CMD MAC (melonDS Wifi::MPCmdMAC)|
|0x2e|`u8[6]`|PictoChat: sender MAC|
|0x34|`u8[6]`|PictoChat: sender MAC (again)|
|0x3a|`u16`|Unknown: counter|
|0x4a|`u16`|PictoChat: Message content offset|
|0x4e|`u8[0xa0]`|PictoChat: Message data (in 8x8 tiles where each byte represents two pixels as nibbles)|
|0xee|`u16`|PictoChat: packet sequence number|
|0xf0|`u16`|PictoChat: copy of 0x1a (Resend)|
|0xf2|`u32`|Unknown: constant 0x93ffb8b6|

TODO: 0x4e(PictoChat msg data offset) - 0x18(Ni-Fi header length) == 0x36
-->

### Fiddling

After editing melonDS source code to corrupt the local multiplayer message
buffer in known content locations before sending the message to the FIFO
buffer:

![](../assets/pictochat-msg-corrupt-lork.png)
![](../assets/pictochat-msg-corrupt-lork2.png)

This shows a few important details:

- Messages are not limited to the apparent 228x80 size (there is also message
  content behind the username label in the above screenshots, but the username
  label is on a different graphics layer)
- The ordering of pixels in the messages is not reading order
- Message content is not checked or validated in any way

## Room host

The system that initially joins an empty room appears to become the room host.
This system ends up never sending messages of Ni-Fi type 2 (Reply), while the
packets from other systems after joining an existing room are sent as Ni-Fi
type 2 exclusively.

![](../assets/ws-announce-vs.png)

(Bottom Wireshark window is the room host, top window shows type != 2 Ni-Fi
messages from the system that joined later)

## Drawing format

- The drawings appear to be split into 8x8 sprite tiles. TODO: pixel order
  within tiles + TODO: sprite ordering (likely both reading order).

### Palette

|idx|hex|
|-|-|
|0|transparent (white)|
|1|`#000000` <b style="color: #000000">■</b>|
|2|`#d3cbc3` <b style="color: #d3cbc3">■</b>|
|3|`#eb00eb` <b style="color: #eb00eb">■</b>|
|4|`#fb008a` <b style="color: #fb008a">■</b>|
|5|`#fb0028` <b style="color: #fb0028">■</b>|
|6|`#fb4900` <b style="color: #fb4900">■</b>|
|7|`#fba200` <b style="color: #fba200">■</b>|
|8|`#e3f300` <b style="color: #e3f300">■</b>|
|9|`#82fb00` <b style="color: #82fb00">■</b>|
|a|`#10fb20` <b style="color: #10fb20">■</b>|
|b|`#00fbba` <b style="color: #00fbba">■</b>|
|c|`#00c3fb` <b style="color: #00c3fb">■</b>|
|d|`#0079fb` <b style="color: #0079fb">■</b>|
|e|`#0030fb` <b style="color: #0030fb">■</b>|
|f|`#2800fb` <b style="color: #2800fb">■</b>|

## More interesting sources / link dump

- <https://www.problemkaputt.de/gbatek.htm#dswifiieee80211frames>
- <https://www.problemkaputt.de/gbatek.htm#dswifinintendobeacons>

## Unsure/notes

- Is the endianness of the DS properly emulated?
- Is the Ni-Fi magic value also present in physical frames or is this something
  MelonDS did?
- The DS implemented WEP(?) encryption for connecting to home network
  routers/APs, but is this encryption also used when the WiFi module is used in
  local multiplayer mode? Does this even matter inside the emulator?

## TODO:

- message reassembly field? (how does pictochat know which part a message index is)
- are message-resends required or can the packets be dropped?
- user identifier / login / announcement procedure?
- actual message format
    * resolution
    * pixel ordering
    * palette color indices (pixels are 1 nibble)
- what types of pictochat packets are there? (i.e. how are room join/leave
  events broadcast?)
- `pictochat.msg_type in {10, 24, 86}` message reassembly in dissector