To avoid name clashes, avoid bloating the “music” folder, and make it a bit easier for Pokitto Arcade to create game packets (*.plz) for downloading, I suggest a new place for game data files (includes music files):
SD: /data/<game>/*.*
e.g. SD: /data/Scrambler/Scram_01.raw
The <game> folder should be exactly the same as the game binary file (*.bin or *.pop) name
I have included the version number in the .pop/.bin files of some of my games, and I would like multiple versions to share the same data files. If I were to convert the save data location for Planet Escape to this new format, I would want people to be able to play their old saves if I haven’t updated the save version. What’s the best way to handle that case? If people have to change their games for this new standard anyway, maybe the name of their data folder could be an optional field in the .pop file?
I like that suggestion, yes force everything into the /data/ folder, but allow the creator to name the folder…
I suppose you want to import save files files from old versions? If that is the case I guess you could read from the new location first (“/data/PlanetEscape/.”). If it is not there try to read from the old location (“/music”?). And always write new save files to the new location.
That is not so easy for automated scripts to find it if it can be a person name or a company name.
My concern is how I would specify a data folder name of “PlanetEscape”. That is not what my *.pop files are named, as they all have the version number in the name of the file itself. We could use the .pop’s filename a a default, but if I could optionally define a data subdirectory in the .pop file (or somewhere else), it would help keep a consistent location not dependent on the name of the .pop file itself.
I currently put save data in a “pe_saves” folder at the root of the SD card. I would be glad to use a standard location to put my saves and music data, but I would like more control than it just being put in a folder with the same name as my .pop file.
It would also not be easy to find the data location if a .pop/.bin file is renamed. Do you expect any script other than an initial script creating the *.plz file to need to know the folder? If it’s just upon initial creation of the bundled file, it could read metadata from the .pop file or some other standard we could decide on.
While it sounds as a good idea to put the data folder path in the pop file, it might be even better to make a plz file at the same time as the pop file (by the pop creator tool). The plz format header should contain the list of the data file paths anyway. Now the plz is created as a separate phase(by me) after the pop and data files have first been pushed to the GameDisk repository.
Might even be worthwhile to look into incorporating the game disk into the PGS at some point. That way we could have a web-based tool for submitting a game which would create the appropriate plz files. The server could have methods for browsing/downloading files.
That sounds good, having the .plz be automatically created alongside the .pop would be convenient for everyone. It could just be an optional field the user enters in the pop creator tool to specify their data directory. If a game doesn’t use a data directory, then it could be left blank.
I think we should avoid making the PGS a jack-of-all-trades, and leave it focused on being a game server. I think it would be better if Pokitto Arcade pulled from a dedicated web repository like I think it’s doing now. It would be nice to have a web-based submission form for submitting games to Pokitto Arcade, but generally speaking I don’t think the PGS should try to serve web pages.
Yes, it might be good to have separate servers.
So that would need an update from @FManga to the POP tool:
In the mean time I can modify my python script, which creates the PLZ-file, to automatically check if the data files are found in: SD:/data/<game>/*.*
Personally I like the idea of games having their own folder and ideally keeping only to that folder unless given permisson to do otherwise.
If software did that more often, there wouldn’t be so much junk in %APPDATA% and the registry (and presumably Linux has a similar problem with things getting dumped in the home directory).
However, one downside to the folder and the .pop file having to have matching names is that things might get awkward if two games have the same name.
Another option would be to introduce some kind of archive format (akin to .zip, .tar et cetera) that stores all the game files as multiple virtual files/a virtual filesystem inside a single physical file. It would be more work, and it would mean more overhead, but it would theoretically mean that the name of the file is irrelevant.
This is precisely what I would have suggested.
Though another option would be to have a separate ‘save migration’ tool.
Assuming PGS is the server that directs data between games, I agree.
Game message tunnelling and game uploading/downloading should be separate services run on separate servers.
Something like that would have to be curated, though I don’t think there would be too much issue.
If there were a way to associate an upload with a forum account then we could make it so that trusted forum users don’t have to have their uploads vetted, and could possibly have some of the upload information auto-filled. Ideally it would be nice to associate a unique ID with each game (and/or version) as well, but all that would take a fair bit of infrastructure.
I’m all for a folder-per-game containing it all, at the very least for binary files. It’s much easier to clean that way.
I always found it absurd to have all the musics located in the same folder, and while the “we could have a player that would play them all” was promising, it was never implemented (you’d need a standard way to tell what the sample is for all the raw files anyway, and you might not want to play some technical raw files since they could be atmosphere or have other purposes)
However, that logic isn’t that great regarding save files. That’s also why there is the Registry, APPDATA and Home folder - to keep the user-generated files off what could be recreated after reinstall. Technically, using Cookies / EEPROM is very close to that solution already. Uninstall the game, your saves/highscores are still on the device!
Regardless of the solution, if two different games have the same names, conflicts will arise - be it in cookies, additional files or even popfiles. It a game shall be published, some care must be taken in order to make sure the names don’t conflict.
About the versions, that’s an annoying one. Is there any interest to keep two different versions of the same game?
If you want to import the previous save, the best way is, I think, to keep the version somewhere inside the save (as a magic value or something), and warn the player when you’re going to import that save, check for errors if possible and delete or upgrade the new save file. If someone wants to play a previous version after playing a new one, they’re already doing something off the roads imho - and warning them before you’re importing the save file would allow them to ask you what would be the best way to do so.
That’s a double edged sword though.
If you were expecting an uninstall to also wipe all of your save data and settings and it doesn’t then you’re left with clutter on your machine. Unfortunately that happens all too often.
If every program kept its settings and save data next in a folder in the same path as the executable then it would be a lot easier to keep track of them and know when the software is fully deleted. That would also make it possible to just avoid deleting those folders when you deleted the rest of the software.
The other advantage to that arrangement is that you could isolate the program by pretending its own folder is the root folder and nothing else exists, thus preventing it from touching anything outside of its own root folder. (Almost the opposite of how Unix/Linux can mount an external storage device as a subfolder.)
Admittedly the ‘home folder’ is one way to solve the issue of having multiple user accounts, but there’s no reason there couldn’t be separate subfolders for each user, e.g. <program root>/saves/<user>. As long as the program has a means to query the current user’s name/id, it would be quite straightforward.
In regards to Pokitto, it ought to be possible to have some de facto or standardised arrangement like so:
<game-name> (or <game-id>)
saves folder (or e.g. savedata)music foldersound folder (or e.g. sfx)data folder
game.pop filegame.bin filesettings fileThen to delete a game and retain its save data, you’d just avoid deleting the saves folder, and possibly the settings file (I presume no game is going to need more than a single file for settings).
There could even be some kind of path list specifying which files/folders to delete to achieve a particular level of uninstalling (i.e. full uninstall versus a partial uninstall that retains save data and settings).
Definitely.
Even if there were to be some kind of music player, you could just feed it a path list, or if we followed a system like the one I outlined above, the music player could just open the games folder and check every folder in that for a music subfolder.
That wouldn’t solve the format issue, that would require more standardisation, but it would solve the issue of how a music player can locate the music a game uses.
Failing that, a game could again provide some kind of file that identifies where its sound is located and what format the files are in.
Whether this is an issue or not depends on the context.
From the user’s point of view, they could name the game folder whatever they want and they’d know what it was.
That only becomes an issue when game loaders and online game repos get involved.
One possible way to solve that issue would be to (again) provide some kind of manifest/metadata file that identifies the game, in which case only the name/id provided in the manifest would have to be uniquely identifying. (A UUID/GUID might be one option there.)
As I said before, ideally we’d have some sort of central game registry that games are registered with and given a unique id as a result of registration, but that requires a lot of infrastructure. Even if we did end up doing that, it could coexist alongside other games if games from the registry were kept in a separate folder to games that aren’t in the registry. (E.g. /games for non-registry games and /registry for registry games.)
Testing might be one exception, though I don’t suppose many people are going to load a game onto their SD card during testing, they’re more likely to load it directly via USB.
This one is the more complex problem.
Personally I think it would be enough to either give the user instructions to copy their old save into the folder of the new game (or even install the new version into the same folder that the old version was using) - that should be enough to give the new version/game easy access to the old save, and how the game imports the save from there is an implementation detail (i.e. it’s up to the game designer to decide how to recognise the file and what to do with it - provide an ‘import’ option on the menu, import automatically, give the player warnings et cetera).
Alternatively, a developer could provide some kind of migration tool that just asks the user to select a source and destination and handles the rest. It would only need to be run once.
If the game knows how to convert the file after moving it then some sort of standardised “move file from source to destination” tool (akin to Linux’s mv and Windows’ move) would be enough to do the actual moving, if for some reason the person can’t get to another device that would allow them to move and copy files (e.g. a desktop).
I like having a standard layout for saves/. If it’s the same for each game, it’ll become a good habit of saves maintenance.
The “uninstallation” file is an interesting idea ! we should look into it.
game.bin/pop, really? is as bad as text.txt. Especially since some “games” aren’t even games 
game-id isn’t a human readable one, like a UUID/GUID.Besides the saves folder and pop/bin, any kind of fixed layout will not give any benefits to users. I suggest we leave the rest to developers.
maps/ folder!).Saves and settings are important locations for users who want to find them, which, I think, is a lot more than users looking for any other kind of files; so I do think they should have a fixed path - especially if we want to have a tool for transferring them. I agree for having
The settings file could also be in the saves folder, actually!
Everything else -including audio- is game-specific. Imposing a fixed layout for them doesn’t serve much, as long as it’s still easy to find the saves and settings.
Why giving audio a special treatment? I really don’t understand. Unless there is an effort to use them outside, it’s a pointless constraint.
game.pop/bin, ew! It’s almost as bad as text.txt . Let’s keep the actual game name, it’d be less confusing this way. Especially if we have a game-id that isn’t relevant.
Regarding the game name, GUID/UUID won’t save you from user being confused when they see two games with the same exact name but wildly different. Remember that the RTS for Dune is called Dune2 in order to avoid confusion with another Dune game, despite the games not being related outside of the Dune settings! So really, we should be careful when naming games. Also, having a registry of games/apps has other interesting uses (including having a nice page, or a checklist for people like me who are extremely behind in term of “I tested this game on the pokitto”)
I probably should have written that as <game>.bin.
The intent was not that it would have to be called game, that was just to illustrate where the executable files would go - i.e. in the root.
Though admittedly, it would avoid the issue of trying to identify what the executable file is called, or indeed which executable file to run if the folder contains more than one.
Again, some kind of manifest file that contains all the metadata for a game would help, because that manifest could always have the same name, and it could tell people (or machines) what the actual executable is called. That extra layer would slow things down a bit though, because the file has to be read/parsed before the game is located.
That was for games in the registry folder, not the ordinary games.
If there were some kind of registry folder managed by the download tool (as opposed to the folder containing the manually installed games), there would likely be some kind of manifest file that provided the names of the games. You’d already be downloading them over the network rather than adding them manually, so the download tool could handle all the bookkeeping for you.
If you go for a human readable name over some kind of numerical ID then it becomes a lot harder to uniquely identify games. If you assign each one a different ID, then it doesn’t matter if you have 10 games all named ‘Snake’ or ‘Minesweeper’ because each will have a different ID.
Nintendo was doing that at least as far back as the GBA - each cartridge has an ID number on the bottom right corner. E.g. AGB-BYOP-UKV is the code for “Yu-Gi-Oh! Day of the Duelist” (or at least the European version, I expect the Japanese and American versions used different suffixes, and possibly different ID codes).
Figuring out how the games differ is still the user’s problem - that’s not what the ID number solves.
The purpose of the ID number is that:
:P)If a user needs to figure out how to tell two games apart, they’re going to have to look at the other metadata, e.g. author, genre, preview image. It would probably be better to focus on making that information more readily available than requiring the user to figure out how to avoid putting two games with the same name in the same folder.
It could be data/maps instead, but that’s less of a concern than music or sound effects because that genuinely is likely to be very game-specific.
The main point was to illustrate how everything could comfortably sit in a single folder without needing to touch anywhere else in the file system, though I don’t think having some kind of standardised (or at least mostly standardised) layout would do any harm. In fact, I think it would do more good than harm.
How many different audio formats are actually in use?
(Note that two files with different bitrates both using PCM would be the same format.)
In cases where it’s just raw PCM played at different bitrates, retrofitting some bitrate metadata would theoretically be possible if there was a need or reason to do so.
If there were to be some kind of music player like the one you mentioned then it would be useful if it had an easy way to know where to find the music files.
Some kind of ‘music manifest’ file would be another option, but parsing a music list for each game would slow things down compared to just peeking inside the music folder and seeing what’s there.
At any rate, it’s better from an organisational standpoint than just dumping everything in the root. I don’t see the harm in everyone putting their music in a music folder (assuming they are keeping their music in separate files instead of embedded in the game binary).
Unless there’s some significant benefit to just dumping everything in the root folder or being able to choose a different name, e.g. it somehow being less computationally expensive, then I don’t see the harm in trying to get more games following a standardised structure.
On the other hand, if everyone just dumps their files wherever they want and every game follows a different system them then it’s just going to complicate any attempt to keep track of what’s what.
I’m not saying ‘ban people from using other approaches’, but if everyone endeavoured to follow a similar pattern to the point where it became a de facto standard then there may be some advantages to be had.
It would, but unfortunately it’s also a lot of work to get up and running.
Or at least it would be to do it ‘properly’ - an organised database with all the relevant metadata, some kind of online system for game registration, a system for downloading files, and some kind of Pokitto+ program for downloading without an external computer.
I agree that having a structure like "SD:/Programs/<Game name>/data/" &"saves/" etc. sounds good. However that would require changes to the Kraken loader and PokArcade which are based on the current "/<order num + genre>/" structure, like "/2Arcade/".
As Pokitto being a hobby project, there is always a risk that fun/work -ratio 
of doing this kind of change is just too small, and it won’t be ever made. So what could be a first step towards that direction, that would not require too much changes? What are the first goals? My original motivation was to avoid bloating of the “music” folder and make it easier to make plz-packets by a script. Then there were good points about the save files, etc.
Then let’s not patronize developers more than necessary (i.e. bin/pop, all of their files inside the same base folder and saves/settings files in predictable paths if they want them to be taken care specially).
Their game folder being a mess or organized is their responsibility, as long as it stays in this folder. Most people won’t check there anyway.
And if a developer wants people to listen to their music, they can implement a in-game player (easiest I think, and an excellent bonus which always reproduces accurately how the music sounds (no pun intended) in the game, fits perfectly proc-gen music like @tuxinator2009 's Incipient) and/or just release an album (which then takes standardization effort whatever the path it is taken, shall it be played on the pokitto itself).
In any case, yeah, I do dislike having my root folder being a mess if every game had their files located inside their own folder it’d be best for everyone.
@Hanski, is the Kraken loader imposing some kind of structures?
/music folder)/games/<game>/ which contains the .bin/.pop, all the files/saves/settings/musics in a contained fashion./2Arcades and other related folders having shortcuts to the pop files.It isn’t, but using it as a “loader” implies that it is easy to browse and load games. Not that you need to browse to the folder of each game to see the title screen and other information, or lose the genre-based structure. Your shortcut suggestion could do it but it need some changes in Kraken and PokArcade.