Extensible Loader

do you know of a way to execute from ram?
my concern is if ech time you start the device your overiding the flash multiple times (overiding bootloader, overiding with file system, overiding with emulator or game)

@FManga

I think having a different extension (.pex is fine by me or .pop or .pog - short for pokitto program or pokitto game, or whatever) is the right way to go

As soon as I saw the new topic I was kind of expecting to see a working bin here … but I know we have all been spoilt by your skillz so far :wink:

2 Likes

This sounds great!

A good idea.

I am not sure if I understand your main idea fully, but this how I think it works:

  1. Boot.pex shows the list of files. It can show icon, screenshot, etc. too if the file is *.pex, right(?). For other files, e.g. *.py, *.bmp, it just shows the default icon, if anything.

  2. When you select a list item that is a pex file, Boot.pex flashes it and makes a reset(?). After that the pex file can be run. Normal Pokitto binary files can be converted to pex files in PC, and put to SD card.

  3. When you select a list item that is e.g. bmp file, Boot.pex searches for the PDL that can handle the file, flashes it and the bmp file to rom(no reset needed?), and gives control to the pld program. The pld then shows the bmp image

Does this sound about correct?

MicroPython can either load a py or mpy(bytecode) file to ram, or use frozen rom file (bytecode). The frozen rom file can be executed in-place, no need to load it to ram.

In theory, just copy code to RAM and call it. I’m still experimenting with this, the tricky part is making useful code that fits strict size limits and doesn’t clash with other libraries (bss sections can’t overlap). Right now I’m making a screen mode library that fits in SRAM1 (2kb).

The flash will be written to less than it is now. The only time it writes to flash is once you pick a game. If you select a GB ROM, for example, it will check if the emulator is already in flash before copying it. In that case it would only copy the ROM itself. Once that is done, it will reset and it won’t write to flash again.


Hehe, I think I’ll go with .pop.

It sure is weird making a topic with a bunch of unproven concepts and no bin to show. I just wanted to make sure this is something the community would like before investing too much effort into it.


Kinda.

  • boot.pex will search for the pdls it can use and it lets the user select one. So the first thing you see will be a menu like this: [Python] [Gameboy] [Games] [Text] [Images]. Each being a separate pdl.
  • Once you select a loader, boot.pex will load that PDL into RAM and search for files that it can read.
  • For each file, boot.pex will request an icon/name/screenshot from the selected loader. If there is none, the loader can return a default.
  • Once you select a file, boot.pex will request the loader PDL to load it.
    – In the case of a BMP, I was thinking of not writing anything to flash here. The Image.pdl would simply display the image instead.
    – when you select a PEX file, game.pdl flashes it and resets. There is no special treatment for pex files in boot.pex, they’re loaded by a pdl just like the other files. Yes, pex files are made in a PC and copied to the SD. A regular bin is a valid pex, though the inverse might not be true.
    – when you select an MPY, both it and micropython are copied to flash, then it resets.

MicroPython can’t run source code in rom? :thinking:

can you clearafy what you mean by that, in theory you dont hardly need any ram for screen mode if you draw directly to screen from sd card (direct 4color bitmap for fonts and sprites) ok maybe the a small amount for the pffs buffering stuff
(or just go with 565 sprites (16bit per pixel) would be slower but more colorfull since it could show all preview images in there games pallets)

maybe unrealated but on the interpreted side could you do anything practical with something like a brainfuck thing? that would be very small on the flash?

   #include <stdlib.h>
    char m[9999], *n[99], *r = m, *p = m + 5000, **s = n, d, c;
    main() 
    {
       for (read(0, r, 4000); c = *r; r++)
              c - ']' || (d > 1 || 
              (r = *p ? *s : (--s, r)), !d || d--), c - '[' || d++ ||
              (*++s = r), d || (*p += c == '+', *p -= c == '-', p += c == '>', 
              p -= c == '<', c - '.' || write(2, p, 1), c - ',' || read(2, p, 1));
    }

Like I said, I’m experimenting and nothing is set in stone yet. I haven’t decided on which screen mode we’ll actually use. We can use direct mode, but that limits what we can do with animations. I’ll let someone better with art propose a UI mockup before deciding anything.

I'm actually testing a bytecode interpreter, but it's not meant to be turing-complete.
.code 16
.syntax unified

.global PVCOPY
.func PVCOPY
PVCOPY:
	push {r4, lr}
	ldr r4, =jmptbl
	movs r2, r0
next:	
	movs r3, #3
	ldm r2!, {r0, r1}
	cmp r0, #0
	beq exit
	ands r3, r0
	lsls r3, #2
	ldr r3, [r4, r3]
	bx r3
	
exit:
	pop {r4, pc}
	
PVCOPYSET:
	str r1, [r0]
	b next
	
PVCOPYFUN:
	movs r3, r0
	movs r0, r1
	push {r2, r4}
	blx r3
	pop {r2, r4}
	b next

PVCOPYOR:
	lsrs r0, #2
	lsls r0, #2
	ldr r3, [r0]
	orrs r3, r1
	str r3, [r0]
	b next
	
PVCOPYAND:
	lsrs r0, #2
	lsls r0, #2
	ldr r3, [r0]
	ands r3, r1
	str r3, [r0]
	b next

	.pool
jmptbl:	
	.word PVCOPYSET+1
	.word PVCOPYFUN+1
	.word PVCOPYOR+1
	.word PVCOPYAND+1
.endfunc
	
It runs code that looks like assembly.
.macro SET REG:req, VAL:req
.word \REG
.word \VAL
.endm

.macro CALL REG:req, VAL:req
.word (\REG)+1
.word \VAL
.endm

.macro OR REG:req, VAL:req
.word \REG+2
.word \VAL
.endm

.macro AND REG:req, VAL:req
.word \REG+3
.word \VAL
.endm

.macro END
.word 0
.endm

.align 4
PV_INIT_GPIO:
	// system_LPC11U6x.c: 487
	OR LPC_SYSAHBCLKCTRL, 1<<16

	// 488
	SET LPC_SYSPLLCTRL, 0x23
	
	// 494, 495
	SET PIO2_0, 1
	SET PIO2_1, 1

	// 497 - Redundant? Default is already 0.
	SET LPC_SYSOSCCTRL, 0

	// 498
	AND LPC_PDRUNCFG, ~(1<<5)

	WAIT_MS 250

	// 507
	SET LPC_SYSPLLCLKSEL, 1

	// 508
	CALL toggle, LPC_SYSPLLCLKUEN

	// 511
	CALL poll, LPC_SYSPLLCLKUEN

	// 522
	OR LPC_PDRUNCFG, 1<<7

	// 523
	SET LPC_SYSPLLCTRL, 0x23

	// 522
	AND LPC_PDRUNCFG, ~(1<<7)

	// 525
	CALL poll, LPC_SYSPLLSTAT

	// 528
	SET LPC_MAINCLKSEL, 3

	// 529
	CALL toggle, LPC_MAINCLKUEN
		
	// 532
	CALL poll, LPC_MAINCLKUEN

	// 534
	SET LPC_SYSAHBCLKDIV, 1

	// 556, 559
	AND LPC_PDRUNCFG, ~(1<<10) & ~(1<<8)

	// 560
	SET LPC_USBPLLCLKSEL, 1

	// 561
	CALL toggle, LPC_USBPLLCLKUEN

	// 564
	CALL poll, LPC_USBPLLCLKUEN

	// 566
	SET LPC_USBPLLCTRL, 0x23

	// 567
	CALL poll, LPC_USBPLLSTAT

	// 569
	SET LPC_USBCLKSEL, 0

	// 573
	SET LPC_USBCLKDIV, 1

	// spi_api.c:77
	SET LPC_SSP0CLKDIV, 1
	// spi_api.c:79
	OR LPC_PRESETCTRL, 1

	// spi_api.c:89
	SET PIO0_9, 0x81
	SET PIO0_8, 0x81
	SET PIO0_6, 0x82

// SPI.cpp
	// disable ssp, set lbm, ms, sod, divider to 0
	AND LPC_SPI0_CR1, ~(1<<1) & ~0xD & ~0xFF00

	SET LPC_SPI0_CR0, 0x7

	// set prescaler to 0. spi_frequency:151
	SET LPC_SPI0_CPSR, 0
	
	OR LPC_SPI0_CR1, (1<<1) // enable ssp
// end SPI.cpp
	
	SET ARM_NVIC_ISER, 0x7F
	SET PIO1_31, 0x80
	
	SET LPC_GPIO_PORT_MASK0, 0
	SET LPC_GPIO_PORT_MASK1, 0
	SET LPC_GPIO_PORT_MASK2, ~(0x7FFF8)

	// #define LCD_CD_PORT           0
	// #define LCD_CD_PIN            2
	// LPC_GPIO_PORT->DIR[LCD_CD_PORT] |= (1  << LCD_CD_PIN );
	// GPIO_DIR[0]:
	SET LPC_GPIO_PORT_DIR0, 0x00000004

	// #define LCD_WR_PORT           1
	// #define LCD_WR_PIN            12
	// LPC_GPIO_PORT->DIR[LCD_WR_PORT] |= (1  << LCD_WR_PIN );
	// #define LCD_RD_PORT           1
	// #define LCD_RD_PIN            24
	// LPC_GPIO_PORT->DIR[LCD_RD_PORT] |= (1  << LCD_RD_PIN );
	// #define LCD_RES_PORT          1
	// #define LCD_RES_PIN           0	
	// LPC_GPIO_PORT->DIR[LCD_RES_PORT] |= (1  << LCD_RES_PIN );
	// GPIO_DIR[1]:
	SET LPC_GPIO_PORT_DIR1, 0x01001001
	
	// LPC_GPIO_PORT->DIR[2] |= (1  << 2 );
	// LPC_GPIO_PORT->DIR[2] |= (0xFFFF  << 3);  // P2_3...P2_18 as output
	// GPIO_DIR[2]:
	SET LPC_GPIO_PORT_DIR2, 0x0007FFFC

	SET LPC_GPIO_PORT_PIN0, 0
	
	// #define LCD_RES_PORT          1
	// #define LCD_RES_PIN           0
	// #define SET_RESET LPC_GPIO_PORT->SET[LCD_RES_PORT] = 1 << LCD_RES_PIN; // RST
	SET LPC_GPIO_PORT_PIN1, (1<<LCD_RES_PIN)
	
	// LPC_GPIO_PORT->SET[2] = 1 << 2; // backlight
	SET LPC_GPIO_PORT_PIN2, 1 << 2
	// LCD initialization

	// Reset LCD
	WAIT_MS 10
 	SET LPC_GPIO_PORT_CLR1, (1<<LCD_RES_PIN)
	WAIT_MS 10
	// (LCD_RD high = write)
 	SET LPC_GPIO_PORT_SET1, (1<<LCD_RES_PIN) | (1<<LCD_RD_PIN)
	WAIT_MS 10
	// driver output control, this also affects direction
	LCD_CMD 0x01,0x11C

	// originally: 0x11C 100011100 SS,NL4,NL3,NL2
        // NL4...0 is the number of scan lines to drive the screen !!!
        // so 11100 is 1c = 220 lines, correct
        // test 1: 0x1C 11100 SS=0,NL4,NL3,NL2 -> no effect
        // test 2: 0x31C 1100011100 GS=1,SS=1,NL4,NL3,NL2 -> no effect
        // test 3: 0x51C 10100011100 SM=1,GS=0,SS=1,NL4,NL3,NL2 -> no effect
        // test 4: 0x71C SM=1,GS=1,SS=1,NL4,NL3,NL2
        // test 5: 0x
        // seems to have no effect... is this perhaps only for RGB mode ?

	// LCD driving control
	LCD_CMD 0x02,0x0100
	// INV = 1
	
	// Entry mode... lets try if this affects the direction
	LCD_CMD 0x03,0x1038
	// originally 0x1030 1000000110000 BGR,ID1,ID0
        // test 1: 0x1038 1000000111000 BGR,ID1,ID0,AM=1 ->drawing DRAM horizontally
        // test 4: am=1, id0=0, id1=0, 1000000001000,0x1008 -> same as above, but flipped on long
        // test 2: am=0, id0=0, 1000000100000, 0x1020 -> flipped on long axis
        // test 3: am=0, id1=0, 1000000010000, 0x1010 -> picture flowed over back to screen

	// Display control 2
	LCD_CMD 0x08,0x0808 // 100000001000 FP2,BP2
	
	// RGB display interface
	LCD_CMD 0x0C,0x0000 // all off
	
	// Frame marker position
	LCD_CMD 0x0F,0x0001 // OSC_EN
	
	// Horizontal DRAM Address
	LCD_CMD 0x20,0x0000

	// Vertical DRAM Address
	LCD_CMD 0x21,0x0000
	
// *************Power On sequence ****************
	LCD_CMD 0x10,0x0000
	LCD_CMD 0x11,0x1000
	
	WAIT_MS 10
//------------------------ Set GRAM area --------------------------------
	// Gate scan position
	LCD_CMD 0x30,0x0000 // if GS=0, 00h=G1, else 00h=G220
	
	// Vertical scroll control
	LCD_CMD 0x31,0x00DB // scroll start line 11011011 = 219
	
	// Vertical scroll control
	LCD_CMD 0x32,0x0000 // scroll end line 0
	
	// Vertical scroll control
	LCD_CMD 0x33,0x0000 // 0=vertical scroll disabled
	
	// Partial screen driving control
	LCD_CMD 0x34,0x00DB // db = full screen (end)
	
	// partial screen
	LCD_CMD 0x35,0x0000 // 0 = start
	
	// Horizontal and vertical RAM position
	LCD_CMD 0x36,0x00AF //end address 175
	
	LCD_CMD 0x37,0x0000
	LCD_CMD 0x38,0x00DB //end address 219
	

	LCD_CMD 0x39,0x0000 // start address 0
	WAIT_MS 10
	
	// start gamma register control
	LCD_CMD 0xff,0x0003
// ----------- Adjust the Gamma  Curve ----------//
	LCD_CMD 0x50,0x0203	
	LCD_CMD 0x51,0x0A09	
	LCD_CMD 0x52,0x0005	
	LCD_CMD 0x53,0x1021	
	LCD_CMD 0x54,0x0602	
	LCD_CMD 0x55,0x0003	
	LCD_CMD 0x56,0x0703	
	LCD_CMD 0x57,0x0507	
	LCD_CMD 0x58,0x1021	
	LCD_CMD 0x59,0x0703	
	LCD_CMD 0xB0,0x2501	
	LCD_CMD 0xFF,0x0000	
	LCD_CMD 0x07,0x1017
	
	SET LCD_CD_CLR, (1<<LCD_CD_PIN)
	SET LCD_MPIN,   (0x22)<<3
	SET LCD_WR_CLR, (1<<LCD_WR_PIN)
	SET LCD_WR_SET, (1<<LCD_WR_PIN)
	SET LCD_CD_SET, (1<<LCD_CD_PIN)
	
	SET PALETTEPTR, 0x18E30000
	SET PALETTEPTR+4, 0xFFFF528A
	
	END
Since it's not turing-complete, it calls functions to do things it can't do itself.
.func poll
.align 4
poll:
	movs r2, 1
1:	
	ldr r1, [r0]
	ands r1, r2
	beq 1b
	bx lr
.endfunc

.func wait
.align 4:
wait:
	movs r1, #10
1:	
	subs r1, #1
	bne 1b
	subs r0, #1
	bne wait
	bx lr
.endfunc	

.macro WAIT_MS MS:req
	CALL wait, \MS
.endm
It also calls functions for things that would take up more space in bytecode than in native code.
.func lcdCmd
.align 4
lcdCmd: 				// r0 = cmd<<16 + arg
	lsrs r1, r0, #16 		// r1 = cmd
	uxth r0, r0 	 		// remove cmd from r0

	lcd_clr_cd r3, r4		// CLR_CD. r3 = pin, r4 = CLR0

	// MPIN[2] = CMD<<3
	ldr r2, =LPC_GPIO_PORT_MPIN0
	lsls r1, r1, #3
	str r1, [r2, #8]
	
	movs r1, r2			// r1 = MPIN
	
	ldr r3, =(1<<LCD_WR_PIN)	// CLR_WR
	str r3, [r4, #4]		// [r4, 4] is CLR1. LCD_WR_PORT

	lsls r0, r0, #3			// data = data<<3
	nop
					// SET_WR
	ldr r2, =LPC_GPIO_PORT_SET0
	str r3, [r2, #4]		// [r2, 4] is SET1. LCD_WR_PORT
	
	movs r3, (1<<LCD_CD_PIN)
	str r3, [r2, #0]		// SET_CD. [r2, 0] is SET0.
	
	str r0, [r1, #8]		// MPIN[2] = data (r0)
	
	ldr r3, =(1<<LCD_WR_PIN)	// CLR_WR
	str r3, [r4, #4]		// [r4, 4] is CLR1. LCD_WR_PORT

	nop
	nop
					// SET_WR
	ldr r2, =LPC_GPIO_PORT_SET0
	str r3, [r2, #4]		// [r2, 4] is SET1. LCD_WR_PORT

	bx lr
	.pool
.endFunc

.macro LCD_CMD CMD:req, ARG:req
	.word lcdCmd+1
	.word (\CMD<<16) | (\ARG)
.endm

That bytecode does all the system and LCD setup, leaving things in a ready-to-go state for the rest of the app. Still not sure if it’s a good idea, I haven’t compared it to the current C++ implementation to see if it actually saves space. All I know is: it works. The final bin does everything in system_11U6x.cpp, initializes SPI, initializes the LCD, then writes to the screen in mode 1 (code not included above) while taking up about 1.5kb in total.

That explanation makes the concept much clearer!

That’s handy!

Absolutely. I cannot figure out any downsides compared to a current implementation, but just huge advantages :slight_smile:

iirc it is not currently supported, but I think it is not hard to make it to support. The whole python program is just a huge string. However, in that case MP needs to compile the script to the bytecode and save it to ram (including bitmaps etc.), so we are much limited by the free ram size (MP interpreter needs quite much stack too).

1 Like

I’ve got the recpie for that.

  • 1 (18.25-ounce) package chocolate cake mix
  • 1 can prepared coconut–pecan frosting
  • 3/4 cup vegetable oil
  • 4 large eggs
  • 1 cup semi-sweet chocolate chips
  • 3/4 cup butter or margarine
  • 1 2/3 cup granulated sugar
  • 2 cups all-purpose flour
  • 1 tsp. vanilla extract
  • 2/3 cup cocoa powder
  • 1 1/4 tsp. baking soda
  • 1 tsp. salt
  • 1/4 tsp. baking powder
  • 1 to 2 (16 ounces each) cans vanilla frosting
  • A 20-foot thick impermeable clay layer
2 Likes

I agree.

I think .pbe would be better - Pokitto Binary Executable.

Call it ‘app’ anything and I’m suing.
‘app’ is a horribly overused word.

Recommended reading:
http://gameprogrammingpatterns.com/bytecode.html

I think it depends whether you want plugins to be able to do arbitrary execution or not.
If you wanted to limit plugins to a subset of commands and speed wasn’t a concern then bytecode might be a viable approach.

I think .plp - Pokitto Loader Plugin.

The interpreter would be tiny but the programs would be unrealisitcally huge.
If an esoteric language was going to be used, something stack-based like FALSE would be a better choice.

Do you think a flavor of FALSE would be the way to go, looking into it it’s 1kb for the compiler but aperently it’s all assambly

Could we do a bytecode version of that?

That’s 1KB for the original x86 compiler, which also does zero error handling, so you can’t go by that.
The size of the code ported to ARM would be different, especially if we added a safety net to prevent it doing dangerous things.

A bytecode variation of FALSE would be pretty easy though, I write them for fun when I get bored :P
Technically FALSE is already bytecode-based,
and the instructions it uses are conviniently also printable characters.

I actually already have a basic stack-based bytecode language implemented because I was going to use it as part of a Pokitto project I didn’t get round to doing because I never got round to ordering that 3.3v-5v bridge.
It would need some modifying to be applicable to this loader though.

That recipe can’t be right, there’s nothing fish-shaped in it. :confused:

1 Like

The list of acceptable ‘garnishes’ is twice the size of the actual recipe.
I thought I’d save people the scrolling.

(Personally my favourite garnish is ‘1 cup lemon juice’ - preferably from a combustible lemon.)

1 Like

My intent in the design is, in order to minimize flash usage, whenever possible, everything on flash should serve more than one purpose:

  • The FS API will be used by the loader and games that support it.
  • The initialization routine, which is in a sort of bytecode, could also be used by the loader and games. The interpreter for that is just 32 thumb instructions.
  • The PEX API will be used by the loader to load plugins and to load the actual games. It can also be used by games that want dynamic code loading. If no games want that, at least the parser can be trivially small.
3 Likes

What sort of idea do people have at this point? Something similar to the original loader mockups, or something completely different?

Not any specific layout in mind (mockup looks good to me), but I would like it to have at least the (reduced) screenshot and the title of the game visible. I just noticed with the Jonne’s latest Game Disk, that having 100 games, with no clue about the content of the game, is, well, tiresome :slight_smile:

Btw. I have implemented BMP-RLE decoding in PokittoLib (search for “RLE”). If that is used in the loader PDL, the screenshots in the PEX file can be RLE-compressed.

For an initial release I’d prefer something simpler/minimalistic, because of the amount of work already involved. Once its usable we can add bells and whistles.

I’ll keep it in mind. While I’m not worried about file size, this should reduce IO.

EDIT: Anybody care to make a clock for the Pokitto? I’d like to remove the time setting/display from the loader, but we’d need to have some other way of setting the time.

Here is a couple of ideas…

Amiga style, ‘mode1 - 4 colours’
mockup

iPod Classic style, full colour, probably very slow
loader_mockup_01

Something based on the current loader, only loading the icon/info for the selected file
although only 3 files at a time doesn’t look so good.
loader_mockup_02

3 Likes

You mean a clock as a separate application (bin)? That would be a nice C/C++ exercise for someone!

Maybe remove the SD from the unselected files so more of them can fit at once?

Yup, exactly.