[WIP] Video player

Here is something for everyone to test. It should run at about 15fps, although it is probably capable of running much faster.

movie.zip

It’s a very early stage movie player for pokitto. I think in the future it might be useful for FMV cut scenes.

At the moment, I’m testing the two different SD libraries that come with pokitto lib, just press A or B once the program starts. I have had better speed out of ‘A’, but better sound out of ‘B’.

All comments welcome.

5 Likes

You should notice a performance improvement if you add padding to align all the data in 512-byte boundaries. Reading from the SD is done in blocks of that size. Reading smaller/unaligned buffers involves throwing away the unneeded parts of neighboring blocks.
As an example: if you request it to fill a 700byte buffer, it will actually read 1024. When you request the 700 bytes after that, it will read 1536bytes.

1 Like

I thought I’d look at this again as it’s been a while. For some reason it doesn’t seem to work and I have no idea why. It seems to be getting stuck somewhere in or before the main playing loop, it accesses the file OK and reads the header info etc.

A working video is attached, just unzip it to the root of the pokitto SD card.

My_settings.h

#ifndef MY_SETTINGS_H
#define MY_SETTINGS_H

#define PROJ_SHOW_FPS_COUNTER
#define PROJ_HIRES              0 //1 = high resolution (220x176) , 0 = low resolution fast mode (110x88)
#define PROJ_MODE13             1 // Spinal's mode13 enabled
#define PROJ_SOUND_BUFFERED     1
#define PROJ_ENABLE_SOUND 1     // 0 = all sound functions disabled
#define PROJ_ENABLE_SYNTH 0
#define PROJ_ENABLE_SOUND       1     // 0 = all sound functions disabled
#define PROJ_AUD_FREQ           11025

#endif
/** Mode 13 (110x88x256) by Spinal, video streaming concept */

#include "Pokitto.h"
#include "SDFileSystem.h"
#include "HWSound.h"

Pokitto::Core game;
int myDelay;
unsigned short pal[256]; // assign a 256 entry array to hold the palette
struct VIDEO_HEADER
{
  uint8_t   ID[3];
  uint8_t   format;
  uint8_t   width;
  uint8_t   height;
  uint16_t  sampleSize;
  uint32_t  frameCount;
  uint8_t   FPS;
  uint32_t  frameSize;
   uint32_t   frameOffset;
} vidhead;
char tempText[64];

void PFFS_Video() { // Petit Fat File System

    uint32_t myFile = fileOpen("movie.dat", FILE_MODE_READONLY);
    if( !myFile ){
        fileReadBytes(&vidhead.ID[0], 3); // should be P,O,K
        fileReadBytes(&vidhead.format, 1); // should be 1
        fileReadBytes(&vidhead.width, 1); // should be 110
        fileReadBytes(&vidhead.height, 1); // should be 88

        vidhead.frameSize = vidhead.width * vidhead.height;
        vidhead.frameOffset = ((88-vidhead.height)/2)*110;

        uint8_t temp1[4];

        fileReadBytes(&temp1[0], 2); // should be ??
        vidhead.sampleSize = (temp1[1] << 8) + temp1[0];
        fileReadBytes(&temp1[0], 4); // should be ??
        vidhead.frameCount = (temp1[3] << 24)+(temp1[2] << 16)+(temp1[1] << 8)+temp1[0];

        fileReadBytes(&vidhead.FPS, 1); // should be 1

        myDelay = 1000/vidhead.FPS;

        sprintf(tempText,"ID:%c%c%c,Fmt:%d,W:%d,H:%d\n",(char)vidhead.ID[0],(char)vidhead.ID[1],(char)vidhead.ID[2], vidhead.format, vidhead.width, vidhead.height);
        game.display.print(tempText);
        sprintf(tempText,"%d\n%d\n%d\n",vidhead.sampleSize,vidhead.frameCount, vidhead.FPS);
        game.display.print(tempText);


        unsigned char col[3];
        for(int temp=0; temp<256; temp++){
            fileReadBytes(&col[0], 3);
            pal[temp] = (col[0]>>3) | ((col[1] >> 2) << 5) | ((col[2] >> 3) << 11);
        }
        game.display.load565Palette(&pal[0]); // load a palette the same way as any other palette in any other screen mode
        int sCount=0;

        game.display.print("Music Start\n");
        pokPlayStream();
        int point=0;
        game.display.print("tempSound\n");
        uint8_t tempSound[vidhead.sampleSize];

        bool stillGoing=1;
        game.display.print("tempTime\n");
        uint32_t tempTime  = game.getTime();

        while(stillGoing==1){
            sprintf(tempText,"%d\n",stillGoing);
            game.display.print(tempText);

            uint32_t elapsed = game.getTime()-tempTime;
            if(elapsed > myDelay){
                game.display.print("NextFrame\n");
                tempTime = game.getTime() - (elapsed-myDelay);
                if(game.update()){

                    game.display.print("AudioFrame\n");
                    // the sample buffer fromthe file is a different size to the pokitto
                    if(!fileReadBytes(&tempSound[0], vidhead.sampleSize))stillGoing=0;

                    point = soundbufindex;
                    for(int t=0; t<vidhead.sampleSize; t++){
                        soundbuf[point]=tempSound[t];
                        if(point++ > SBUFSIZE){point=0;}
                    }

                    if(!fileReadBytes(&game.display.screenbuffer[vidhead.frameOffset], vidhead.frameSize))stillGoing=0;
/*
                    if(game.rightBtn()){
                        pokPauseStream();
                        for(int t=0; t<SBUFSIZE; t++){
                            soundbuf[point]=0;
                        }
                        point = soundbufindex;
                        fileSeekRelative((vidhead.frameSize+vidhead.sampleSize)*150);
                        pokPlayStream();
                    }
*/
                } // update
            } // timer
        } // stillgoing
        fileClose();
    } // if myFile
}

bool videoRunning = false;
int main(){
    game.begin();
    game.display.persistence=1;
    //game.setFrameRate(255);

    int temp;
    SDFileSystem sd(/*MOSI*/P0_9, /*MISO*/P0_8, /*SCK*/P0_6, /*CS*/P0_7, /*Mountpoint*/"sd");
    pokInitSD(); // Call init always.

    game.display.print("Press A\n");

    while (game.isRunning()) {

        if(game.update()){
            if(game.aBtn() &! videoRunning){
                PFFS_Video();
                videoRunning = true;
                
            }
        }
    }

    return 1;
}

movie.zip (9.4 MB)

Anyone want to take a look?

You’re initializing SdFs and the pffs?

1 Like

Ah, that seems to have started something. It was never a problem previously though.

[edit] although it crashes a few seconds in…

The latest build using femto, is copy+pasted from the project that was used in the first post, but it crashes after a few seconds, yet the download in the first post plays the same movie clip without problems. :thinking::worried:

OK, restarting this, as I’ve had some ideas recently, namely, use TAS mode. I had posted some tests in discord a while back, but I’m going to add my progress here.

The way my new method works, is to check each line of the image for similarities to the previous line and simpley remove any pixels that don’t change.
Because of the way TAS mode renders the screen, each line is kept and changes made before sending it to the screen. By only saving the pixels that have changed between the previous line and the current line, I hope to save a lot of space without losing any quality.

You can see from the image below, this could remove quite a bit of unneeded data from a frame. Also, the conversion of the palette from RGB888 to RGB565 for use on the Pokitto, will also result in less colours being used, which can also be remove from the frame data.

I plan to then compress the image using ‘RLE’ which should save a bit of space also.
The idea is to make the filesize of a video resemble a sensible size that people might want to use.

Eventually I would like to make a basic copy of the TAS render method and be able to render a video completely independently of a projects screen mode.

[edit] Maybe a mod could move this to a new thread? Thinking about it, it seems odd to continue a 2 year old post.

2 Likes