[Wiki]Everything there is to know about LibHotswap

Wiki
1

LibHotswap is a C++ library for easily loading data from SD into Flash.

With it:

  • there is no need to mess with linker scripts or low-level APIs.
  • the space is reserved in the bin, so you easily know how much space you’re using.
  • you can have multiple, independent data banks.
  • both synchronous and asynchronous loading are supported.
  • flash is verified so that it isn’t rewritten unnecessarily.
  • works on hardware, emulator, and simulator.

How do you use it?

First, pull the latest version of PokittoLib from github.
Once that’s done:

#include <Pokitto.h>
#include <LibHotswap>

// Make a hotswap area for level data
// size = 512 bytes, unique id = 0
using LevelData = Hotswap<512, 0>; 

// Another hotswap for images
// size = 1024 bytes, unique id = 1
using ImageData = Hotswap<1024, 1>; 

// pointers to our data
const uint8_t* levelData = nullptr;
const uint8_t* imageData = nullptr;

// a callback for when data is loaded asynchronously
void onImagesLoaded(const uint8_t *data){
  imageData = data;
}

void init(){
  // load level data immediately
  levelData = LevelData::load("level1.bin"); 

  // load images asynchronously
  ImageData::load("level1Images.bin", onImagesLoaded); 
}
10 Likes
2

This sounds cool, if I’m understanding it right.

How does one create a data binary? Just declare a const in an empty main and compile?

1 Like
3

This is very nice. One can load practically endless amount of new levels (gfx,sfx,tilemaps,etc) from SD during the game!
Few questions:

  • How the bin to be loaded, is created?
  • Are there cpu resources available during asynch loading? I.e. can I show a progress bar etc. ?
  • Is it possible to load executable code? Asm?
2 Likes
4

  • A data binary can be anything: JSON, text, music, etc. Because of that, there isn’t a single way to prepare it. For images, whatever tools you use for conversion would need to output raw bytes instead of a C array. I’m using a FemtoIDE script that packs the contents of a folder into a single bin, maybe I can include that as a built-in on the next FemtoIDE release.

  • You can do anything during async loading, but I haven’t checked how it impacts the framerate. It varies depending on how much caffeine your goblins have been consuming that day. You can’t display a progress bar, simply because there’s no API to get that information. The idea for async loading is that you can load the next part of a level while still playing (imagine a sidescroller where levels are seamlessly streamed in as you play).

  • As for executable code: yes and no. Asm, yes. C/C++, well, GCC does not support position-independent code (-fPIC), so that’s a problem. In my game, I load various small C++ programs into hotswap, then when I want to run one of them I copy it from flash into SRAM2 and call it there. That way I can have various programs in the same position without having a mess of linker scripts.

6 Likes
5

Oh, I see. Something about the “.bin” extension confused me into thinking there was some kind of magic somewhere.

Also, below is what I used previously (based on reading elsewhere in the forums). Is LibHotswap safer and/or faster? Or is it more about the convenience?

#include "SDFileSystem.h"
#include "FATFileHandle.h"

SDFileSystem sd(/*MOSI*/P0_9, /*MISO*/P0_8, /*SCK*/P0_6, /*CS*/P0_7, /*Mountpoint*/"sd");

FILE * fs_fopen( const char *file, const char *mode )
{
	int flags = O_RDWR;
	if( mode[0] != 'r' )
	{
		flags |= O_CREAT;
		//if( mode[0] == 'w' )
		//    flags |= O_TRUNC;
		if( mode[0] == 'a' )
			flags |= O_APPEND;
	}
	else if( mode[1] == 0 )
	{
		flags = O_RDONLY;
	}
	return (FILE *) sd.open( file, flags );
}

// fclose
int fs_fclose( FILE *fp )
{
	if( fp )
		((FATFileHandle*) fp)->close();
	return 0;
}

// fread
unsigned int fs_fread (void *buffer, size_t a, size_t b, FILE *fp)
{
	unsigned int r = ((FATFileHandle *) fp)->read( buffer, a*b );
	if( r == ~0UL ) r = 0;
	return r;
}

template<typename S>
static bool tryGet(const char * fname, S * data) {
	FILE *fp = fs_fopen(fname, "rb");
	if (fp == NULL) {
		return false;
	}
	fs_fread(data, 1, sizeof(S), fp);
	fs_fclose(fp);
	return true;
}
6

That reads things into RAM, while LibHotswap is for data that is too big for RAM.
Loading into Flash is slower and implies wear, so it shouldn’t be abused.

For loading into RAM I recommend you look into File (the third of the mentioned APIs) as that has advantages over SDFileSystem.

2 Likes
7

Are you making a new game? :flushed: Tell us more about your game! Open a new thread, post some screenshots, write a short devlog, tweet about it … We need more information about @FManga 's new game :stuck_out_tongue_winking_eye:

2 Likes
8

Thanks Felipe, this looks really good. I can see a number of extended games being written using this and I wish it existed when I was developing Scopa.

3 Likes
9

More like rewriting one that had to be stopped half-way through. Before, I was streaming data into RAM as-needed and it worked really well on my own Pokittos, but terribly on some other users’ devices. This variation made me have to rethink the whole thing to be less dependent on SD card speed. There’s a lot of catching-up to do, I’ll make a thread once there’s something interesting to show.

2 Likes
10

I suppose that it’s not meant to be used too often as it can eat the remaining flash life cycles of your device very fast. Afaik there are only several thousands of them. If you are using it frequently in a game, I suppose you should warn about it.

11

I looked through your game source code. Are you storing tons of images for frame by frame animations that are just scaled down versions of every card? :flushed: That’s pretty inefficient use of resources, I suppose.

12

For program flash it’s specified as typically 100000 cycles. If you wrote it once per minute continuously, it would be about 70 days.

For EEPROM it’s specified as typically 1000000 cycles. If you wrote it once per minute continuously, it would be about 2 years.

5 Likes
13

Yep … but they aren’t scaled down :grinning:

1 Like
14

Wouldn’t it be cool to make a multipart game of Perseverance trip to Mars, using the rom hotswap? First launcing the rocket from the earth at the right timing, manouvering near Mars, landing on Mars, moving the rover on Mars, sending the drone to a flight on Mars surface,…
Each would be an own small game connected seamlessly to the next :grinning:

3 Likes
15

I was wondering, is there a way to pre-initialize these Hotswap so we don’t have to necessarily rewrite the flash right on the start?

It’s for this case “I want the default sprite/map/musicpatch/etc most of the time, but if the user loads a mod it’ll be easily overrideable”

1 Like
16

When the device is power-cycled, the mod would still be loaded. To go back, you’d need to make the default map a mod as well, at which point the pre-initialization isn’t really that useful.

17

(A map or anything! I was more thinking about musics or sprites than maps actually, things that can be customized by the player, or something like “second combat music events”, etc)

I’m pointing out that having to reflash the same area with the same bytes is useless and could be avoided with such an initial state; but it’s probably beyond the scope of Hotswap, especially since the uses are gonna be sporadic at best (e.g. a default battle music that is transformed into a new one on advanced stages, etc)

18

I think it’s possible to do. For this first release I just wanted to keep things simple to see what real-world problems will arise instead of trying to throw in the kitchen sink.

5 Likes
19

Actually, maybe scratch that. It appears that the latest version of GCC has fixed PIC. I’m still testing, but it opens up some interesting possibilities if it really does work. No more need to copy code to a fixed location for it to run, at least.

6 Likes
20

This is a cool idea. You could have “texture packs” for a game that would swap the graphics. Most games wouldn’t need that, but I could imagine this being really cool for certain ones. The main issue with it is that it would depend on user-made content to be really compelling. If the game creator makes multiple texture themes, then they can just release them in separate bins. But I do think that this could be really cool under the right circumstances.

3 Likes