CadEditor. History of the project

Repository

https://github.com/spiiin/CadEditor

CadEditor is the universal level editor for NES/SMD/SNES/Game Boy games, the introduction of the project was here

Introduction

This is the article about the history of the project. CadEditor is my pet project started in 2012 as the simple level editor for one game, but now it has powerful config script system, that allows it to work with a large number of games (~200 games supported now). The article has a lot of programming terms, but I tried to write it as simply as possible

Post Body

0. My romhacking experience

I got into game research almost immediately after I discovered the opportunity to emulate old console games with the computer. I skipped period when emulators were developed, and popular games were dumped to ROM-files - they were already available on the internet.

At first, I was interested in secrets in games - unused content stored inside ROM-files, codes for a switch to debug mode for testing, password systems, little-known for gamers bonuses and secret places. Most of this material already was found by authors of dumps and romhackers. But after some research, I came across with that not all have been researched yet.

There are some of my first researches (articles in Russian):
Tecmo World Cup password system
Goal! Two password system
Adventure Island 2 random level select system
Battletoads regional version differences
Addams Family doors system

My first real interesting discovery was hidden prizes in New Ghostbusters 2 [NES]. It can be received only if the player will catch the ghost in a certain order. I found it when research code related to enemies behavior. I found the item ("bag with money"), that contains a property with an array of other items. That was a chain of ghost types needed to catch, for an appearance of the bag with money. There is a video with collect all bags:

One of my ideas of game research was building maps of game levels by binary data stored in ROM-file. It was a hard task for me, and I spent several weeks, but I achieved my goal and I was able to get maps for Jungle Book [NES]. But during the research, I understand that I can repeat this process with other games.

I was mostly interested in building level map only, but I this time, I met romhacker Lomax who created level hacks for classic Capcom NES games. He used hex-editor for changing levels. I wrote to him, that I can create a visual editor for changing levels of Chip & Dale Rescue Rangers[NES] (by that moment I already partially researched this game), and he interested in this idea.

Thus, I started working on a prototype CadEditor, in order to give the opportunity to change levels in the game Chip & Dale for him.

1. Chip and Dale and Darkwing Duck Editor

I distributed the first versions of the program through the Russian forum "emu-land", and received many feedbacks with proposals for improvements and found bugs. I gradually added the necessary features - block editor, build graphics from binary data as console PPU does it, object editor and level parameters editor. At the same time, I discovered that Darkwing Duck game [NES] used almost same level format, and even many data are stored at same offsets, so I add this game in the editor too.

At this point, users of editor created hacks for these two games:

2. Capcom games editor

I was interested: if Chip and Dale and Darkwing Duck games so similar, what about other Capcom games?

I researched level format for Duck Tales 1&2, Tale Spin, and Little Mermaid and found out that they are fairly similar also. But, all of them (except Tale Spin – it was identical to Darkwing Duck), has minor differences, which prevent to add it to the CadEditor.

I was thinking about how to add these games to the editor so as to be able to expand it to new games. Obviously, it was necessary to put the data addresses in the config file. But what about different ways to load data, when the format was unknown yet?

One of the possible solution - to embed script language interpreter to the editor, so config files for new games can be written in this language.

I wrote CadEditor in C#, and chose CSScript library for scripting. This library allows writing a script in C# also. It not a maximum short or simple language, but I can move code from editor to config scripts and back without any changes. Of course, first versions of my script system not allowed to describe any games, and part of loading logic was hardcoded inside CadEditor itself.

But using this approach, I added all games listed above to CadEditor:

3. Games editor with prepared block images

I tried to explore the games from other developers and figure out how the levels are described in them. One of the general formulations was - almost all games use blocks (described in various ways) and an array with a game map that contains the indexes of these blocks. This approach could be transferred to the editor - if I do not understand how blocks were built, but simply prepare screenshots of each block at the level, I still can display a map from prepared pictures and allow to change it. In addition, this approach allowed porting game configs from other level editors.

During editor updates I prepared images of blocks from several games from different developers - Jungle Book, Tiny Toon Adventures, Tom & Jerry, Battletoads.

After that, I had the idea of ​​creating an editor that could work with ANY game using configs. For this, it was necessary to study the format of the levels of a large number of games, without understanding into the detailed structure of each. Next versions of the editor were added new descriptive features to the editor.

4. Universal block editor

I gradually added to the editor the ability to describe various block and screen formats. In versions 1.9-1.9.5, I added the ability to describe games in which screens are not described line by line but described per columns. In addition, the I added ability to use not only square blocks but also rectangular. This allowed for version 2.0 to add 32 games to the editor (one level from each for the tests).

Next, I discovered, that with CadEditor using prepared images of blocks, I can display game levels for the Sega Mega Drive platform and add several levels for it (Tiny Toon, Contra Hard Corps, QuackShot). Further, the development of the editor proceeded through research and the addition of new games, most of which did not require any additional efforts — all that was needed was to prepare block images.

However, it was a routine work that required a lot of time, so by this time I had written several scripts to automate actions. I called the project Autocorrupter – a script that allows you to get pictures of each block on the level. This script has also evolved and improved over time. This required writing the code and making changes to the NES Fceux emulator itself (changes are currently included in the latest version of the emulator).

At this point, the interest of users to the editor faded a little, because I was completely absorbed in adding and researching new games and was not engaged in expanding support for already explored games. However, some users shared my ideas and helped me. To version 3.0, the hacker with the nickname Mephisto took the source code of my editor and ported it to create the editor for Thunder Force 3 for the game Sega Mega Drive. He did not use the editor with pictures but wrote code for building blocks for Sega games. I ported his code back to my editor and this gave me the opportunity to describe the blocks for Sega Mega Drive games.

5. Plugin system

The editor at this point was overgrown with a huge amount of code (partially obsolete - written for specific games, and not for a generic editor), and I realized that the project had become so complex that I could not control it soon.

I started to think about how to simplify the code and break the project into separate parts.

The games for NES and SMD used a separate code for building graphics and editing blocks (there are some platform-specific differences - so editing happens a little differently). Therefore, I decided to add a plugin system to the editor - the common code used for any game remained in the editor's core, and things specific to the platform and the game were put into separate assemblies (in the form of DLL files), which the editor loaded after the game settings file was opened.

For example, when opening the Chip & Dale [NES] game settings, the editor loaded the plug-ins for building NES graphics, as well as the door parameter editing plug-in specific for this game.

For Contra Hard Corps [SMD] – the editor loaded the plugin for building SMD graphics, and a special background editor, which is present for all SMD games:

At the same time, the editors of screens, objects, and blocks always remained in the editor's core. This allowed to improve the architecture and reduce the amount of code in the main project.

6. Checking of different NES-games.

I also found a large class of games that use the same block format of 17 bytes - 16 tiles and 1 byte of the palette. Later, I realized that this is a natural description for the NES platform. I added support for building such blocks, which allowed us to once again go through the games that were described with pictures of blocks, and rewrite them using this blocks. Here are some examples of configuring games with such blocks.
(in Russian)
Teenage Mutant Ninja Turtles 2
Power Blade 2
Description of this types of blocks

Here is a list with some games, using blocks 16+1:

• Power Blade 1-2 (Natsume)
• Adventure Island 2-3 (Hudson soft)
• Yo-Noid (Now production)
• Jackie Chan Action Kung Fu (Now production)
• Contra Force(Konami)
• Batman Returns (Konami)
• Super C (Konami)
• Monster in My Pocket (Konami)
• Bucky O'Hare (Konami)
• Jackal (Konami)
• Teenage Mutant Ninja Turtles 1-3 (Konami)
• Alien 3 (Probe entertaiment)
• Battletoads (Rare)
• Battletoads & Double Dragon (Rare)
• Darkman (Ocean software)
• Many others

To search for blocks in the ROM file, I came up with an algorithm that allowed the search for standard blocks automatically, analyzing the video memory in the emulator. This has greatly accelerated the search process. This algorithm is implemented in the form of a Lua script called BlockFinder for the Fceux emulator. In a pair of scripts, Autocorrupter and BlockFinder allowed to adding support for new games to the editor very quickly.

With the help of new tools and update of the editor for the description of the blocks, a large number of games have been added. User Lancuster connected to the project, and he added all levels for those games, I created only one test config. He found more than 100 games with formats similar to those already added and described them. The total number of games that support the editing of screens and blocks is close to 200.

Resources

https://spiiin.github.io/CadEditor/
I collected ~60 posts and articles about game research and CadEditor (mostly in Russian).

Patreon
Itch.io
Emu-land forum theme related to CadEditor
Romhacking.net forum theme related to CadEditor

Roadmap

Future

The system of configs and plugins of the CadEditor, as well as tools for searching for blocks in ROM files, allows users to describe almost any game now.

However, in addition to the modules of the editor of screens and blocks, in order to conveniently change the games, it is also necessary to add an object editor. The editor supports a generic description for lists of objects, but most games are still waiting to be added to it.

In addition, most of the research concerned only NES games, although similar search principles can be applied to any platform with tile graphics — Sega Mega Drive, SNES, Game Boy, Game Boy Advance, Sega Master System, old PC games.

An interesting feature is the use of an editor with fully rebuild the game from sources (working with asm files). The plug-in editor system allows you to implement an assembly system using external utilities, including assembler, export, and data compression.

But my main goal, above all, is to preserve in the editor the history and culture of old games, as well as research and documenting the solutions used by developers. That is why I spend my time on this project.

Series Backlinks

• CadEditor - Universal level editor for NES/SMD/SNES games
• CadEditor. Support levels with mixed screens
• CadEditor. Release 5.1
• CadEditor - Two new plugins
• CadEditor. Release 5.2